Jul 21 13:42:10.816471 ip-10-0-129-212 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 21 13:42:10.816692 ip-10-0-129-212 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 21 13:42:10.816785 ip-10-0-129-212 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 21 13:42:10.817264 ip-10-0-129-212 systemd[1]: Failed to start Kubernetes Kubelet. Jul 21 13:42:22.410355 ip-10-0-129-212 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 21 13:42:22.410375 ip-10-0-129-212 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot be893f52262f437d9701e554752af73c -- Jul 21 13:43:58.714031 ip-10-0-129-212 systemd[1]: Starting Kubernetes Kubelet... Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.166414 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171270 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171286 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171289 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171292 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171296 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171299 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171302 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 13:43:59.174535 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171305 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171309 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171311 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171314 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171317 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171319 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171322 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171325 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171327 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171332 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171335 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171340 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171344 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171353 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171356 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171359 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171363 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171366 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171368 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 13:43:59.189114 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171372 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171375 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171377 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171380 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171383 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171386 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171388 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171457 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171460 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171463 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171466 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171468 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171472 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171491 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171684 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171711 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171916 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171921 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.171924 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172115 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 13:43:59.190931 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172132 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172134 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172138 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172328 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172345 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172348 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172541 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172557 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172560 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172564 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172777 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172970 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172987 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172990 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.172993 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173162 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173165 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173167 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173170 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173172 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 13:43:59.193851 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173175 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173179 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173181 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173184 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173186 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173190 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173192 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173195 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173197 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173202 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173206 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173208 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173211 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173214 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173216 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173219 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173221 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173224 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173226 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 13:43:59.208790 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173229 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173233 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173235 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173239 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173242 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173245 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173247 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173250 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173253 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173260 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173263 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173268 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173271 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.173274 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175864 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175873 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175877 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175880 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175883 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175888 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 13:43:59.221255 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175891 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175894 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175897 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175901 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175904 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175907 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175910 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175912 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175915 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175918 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175920 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175924 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175926 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175931 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175935 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175938 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175941 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175944 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175946 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 13:43:59.223188 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175949 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175952 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175956 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175960 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175963 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175966 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175970 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175972 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175975 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175978 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175980 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175983 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175986 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175989 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175991 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175994 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175997 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.175999 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176002 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176004 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 13:43:59.225672 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176007 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176009 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176012 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176014 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176020 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176023 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176026 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176028 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176031 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176033 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176036 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176038 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176041 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176044 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176048 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176051 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176053 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176056 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176059 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176061 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 13:43:59.296027 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176064 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176066 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176069 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176072 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176074 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176078 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176083 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176086 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176089 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176092 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176095 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176097 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176102 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176105 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176107 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176110 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176114 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176117 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176119 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 13:43:59.311886 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176122 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 13:43:59.260970 ip-10-0-129-212 systemd[1]: Started Kubernetes Kubelet. Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176124 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176127 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176129 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176132 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176135 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176137 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176141 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176144 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176146 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176149 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176151 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176153 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176156 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176158 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176570 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176575 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176578 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176581 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176593 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 13:43:59.521898 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176596 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176599 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176601 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176604 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176607 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176609 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176612 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176615 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176617 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176623 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176625 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176628 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176632 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176636 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176639 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176642 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176644 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176648 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 13:43:59.535594 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176652 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176656 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176659 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176661 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176664 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176666 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176669 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176672 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176674 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176677 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176679 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176682 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176684 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176687 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176722 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176725 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176728 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176730 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176733 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176735 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 13:43:59.536295 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176738 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176740 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176743 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176747 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176749 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176752 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176754 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176757 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176759 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176762 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176764 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176767 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176770 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176774 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176776 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176779 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176781 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176784 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176786 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176789 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 13:43:59.536991 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176791 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176794 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176796 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176799 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176801 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176805 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176809 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176818 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176821 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176823 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176826 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176828 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176831 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176833 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176836 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176840 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176843 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176845 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176848 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 13:43:59.537571 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176851 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176853 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176857 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176859 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176862 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176864 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176869 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176872 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176874 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176877 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176879 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176882 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176884 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176887 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176890 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176898 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.176901 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.176977 2567 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.176989 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177007 2567 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177012 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 13:43:59.538202 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177021 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177025 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177029 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177034 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177037 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177040 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177044 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177049 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177052 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177055 2567 flags.go:64] FLAG: --cgroup-root="" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177058 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177061 2567 flags.go:64] FLAG: --client-ca-file="" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177064 2567 flags.go:64] FLAG: --cloud-provider="external" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177067 2567 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177074 2567 flags.go:64] FLAG: --cluster-domain="" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177077 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177080 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177086 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177089 2567 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177160 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177169 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177173 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177177 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177181 2567 flags.go:64] FLAG: --contention-profiling="false" Jul 21 13:43:59.538955 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177186 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177189 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177193 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177197 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177203 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177206 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177209 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177212 2567 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177215 2567 flags.go:64] FLAG: --enable-server="true" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177219 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177226 2567 flags.go:64] FLAG: --event-burst="100" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177230 2567 flags.go:64] FLAG: --event-qps="50" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177233 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177236 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177240 2567 flags.go:64] FLAG: --eviction-hard="" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177244 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177247 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177250 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177253 2567 flags.go:64] FLAG: --eviction-soft="" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177256 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177260 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177263 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177266 2567 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177269 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177272 2567 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 13:43:59.539652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177275 2567 flags.go:64] FLAG: --feature-gates="" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177279 2567 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177282 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177290 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177294 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177298 2567 flags.go:64] FLAG: --healthz-port="10248" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177301 2567 flags.go:64] FLAG: --help="false" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177304 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177307 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177311 2567 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177314 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177321 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177325 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177328 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177331 2567 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177334 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177337 2567 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177340 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177344 2567 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177347 2567 flags.go:64] FLAG: --kube-reserved="" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177350 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177353 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177356 2567 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177359 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 13:43:59.540480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177362 2567 flags.go:64] FLAG: --lock-file="" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177365 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177368 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177372 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177378 2567 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177381 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177384 2567 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177387 2567 flags.go:64] FLAG: --logging-format="text" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177390 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177393 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177396 2567 flags.go:64] FLAG: --manifest-url="" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177399 2567 flags.go:64] FLAG: --manifest-url-header="" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177405 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177408 2567 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177413 2567 flags.go:64] FLAG: --max-pods="110" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177416 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177419 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177422 2567 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177426 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177430 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177433 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177438 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177446 2567 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177449 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177453 2567 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 13:43:59.541225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177456 2567 flags.go:64] FLAG: --pod-cidr="" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177459 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177465 2567 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177468 2567 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177471 2567 flags.go:64] FLAG: --pods-per-core="0" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177474 2567 flags.go:64] FLAG: --port="10250" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177477 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177480 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01ad6a84f6b97914f" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177483 2567 flags.go:64] FLAG: --qos-reserved="" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177486 2567 flags.go:64] FLAG: --read-only-port="10255" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177489 2567 flags.go:64] FLAG: --register-node="true" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177492 2567 flags.go:64] FLAG: --register-with-taints="" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177496 2567 flags.go:64] FLAG: --registry-burst="10" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177499 2567 flags.go:64] FLAG: --registry-qps="5" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177502 2567 flags.go:64] FLAG: --reserved-cpus="" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177505 2567 flags.go:64] FLAG: --reserved-memory="" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177509 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177512 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177515 2567 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177518 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177523 2567 flags.go:64] FLAG: --runonce="false" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177528 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177531 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177534 2567 flags.go:64] FLAG: --seccomp-default="false" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177537 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 13:43:59.541946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177540 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177543 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177547 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177552 2567 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177556 2567 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177559 2567 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177562 2567 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177565 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177569 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177572 2567 flags.go:64] FLAG: --system-cgroups="" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177574 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177583 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177586 2567 flags.go:64] FLAG: --tls-cert-file="" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177589 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177593 2567 flags.go:64] FLAG: --tls-min-version="" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177596 2567 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177599 2567 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177602 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177605 2567 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177609 2567 flags.go:64] FLAG: --v="2" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177613 2567 flags.go:64] FLAG: --version="false" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177618 2567 flags.go:64] FLAG: --vmodule="" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177622 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.177625 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177781 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 13:43:59.542649 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177786 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177789 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177792 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177795 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177799 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177801 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177804 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177807 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177810 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177815 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177819 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177821 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177824 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177827 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177830 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177832 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177835 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177837 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177840 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177862 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 13:43:59.543317 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177867 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177870 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177873 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177876 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177880 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177882 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177885 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177889 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177893 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177896 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177900 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177903 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177905 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177908 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177910 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177913 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177916 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177919 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177922 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 13:43:59.543920 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177924 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177927 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177931 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177935 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177938 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177940 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177943 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177946 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177948 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177951 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177954 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177957 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177959 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177962 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177965 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177967 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177970 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177972 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177975 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177978 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 13:43:59.544549 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177980 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177983 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177985 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177988 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177990 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177993 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177995 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.177998 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178002 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178012 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178016 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178019 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178022 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178025 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178030 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178034 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178037 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178039 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178043 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178045 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 13:43:59.545182 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178048 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178050 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178053 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178056 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178058 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178062 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178064 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178067 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178070 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178074 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178077 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178080 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178083 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178086 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178088 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178091 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178093 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178096 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 13:43:59.546007 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.178099 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.178843 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.186564 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.186583 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.187476 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.187486 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187579 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187587 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187592 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187596 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187600 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187605 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187609 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187614 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187617 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 13:43:59.546615 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187621 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187627 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187631 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187635 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187639 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187643 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187648 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187651 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187655 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187659 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187664 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187667 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187674 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187679 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187684 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187689 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187693 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187716 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187727 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 13:43:59.547233 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187731 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187736 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187741 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187745 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187749 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187752 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187757 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187761 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187764 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187770 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187774 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187779 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187784 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187788 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187792 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187797 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187801 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187805 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187810 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187814 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 13:43:59.547866 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187818 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187822 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187826 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187830 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187834 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187838 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187841 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187845 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187849 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187853 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187859 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187863 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187866 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187872 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187878 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187882 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187885 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187890 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187893 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187897 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 13:43:59.550190 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187900 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187904 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187908 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187911 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187915 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187919 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187924 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187928 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187935 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187939 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187942 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187946 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187950 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187955 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187959 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187962 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187966 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187969 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187973 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187976 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 13:43:59.550788 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187980 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187984 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187991 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187995 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.187998 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188002 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188006 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188010 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188013 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188017 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188021 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.188029 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188198 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188205 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188210 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 13:43:59.640577 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188214 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188218 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188222 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188226 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188230 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188235 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188240 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188244 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188247 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188251 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188255 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188259 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188263 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188267 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188270 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188274 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188277 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188281 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188285 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188288 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 13:43:59.641202 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188293 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188297 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188300 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188304 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188307 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188311 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188315 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188318 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188322 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188326 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188330 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188334 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188338 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188341 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188345 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188349 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188355 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188361 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188365 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188369 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 13:43:59.641842 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188373 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188376 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188380 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188384 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188387 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188390 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188394 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188398 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188402 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188405 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188409 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188413 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188421 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188425 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188429 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188433 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188436 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188440 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188443 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 13:43:59.642458 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188447 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188451 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188454 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188458 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188462 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188466 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188469 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188473 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188476 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188480 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188485 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188489 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188492 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188498 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188503 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188506 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188510 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188514 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188517 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188521 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 13:43:59.643067 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188524 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188529 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188532 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188536 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188539 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188544 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188548 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188551 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188555 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188559 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188563 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188567 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188571 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188574 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188579 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188582 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 13:43:59.643645 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:43:59.188586 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 13:43:59.644263 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.188593 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 13:43:59.644263 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.188782 2567 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 13:43:59.644263 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.195588 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 13:43:59.644263 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.197327 2567 server.go:1019] "Starting client certificate rotation" Jul 21 13:43:59.644263 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.197428 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 13:43:59.644263 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.197500 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 13:43:59.644263 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.200252 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 13:43:59.644263 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.204406 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 13:43:59.644263 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.217016 2567 log.go:25] "Validated CRI v1 runtime API" Jul 21 13:43:59.644263 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.219662 2567 log.go:25] "Validated CRI v1 image API" Jul 21 13:43:59.644263 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.220468 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 13:43:59.644263 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.223430 2567 fs.go:135] Filesystem UUIDs: map[1a49560a-42b9-4b16-882c-6f5f4338f654:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f01cc99e-a2f9-4184-a389-2485f9649eb8:/dev/nvme0n1p4] Jul 21 13:43:59.644669 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.223447 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 13:43:59.644877 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.229544 2567 manager.go:216] Machine: {Timestamp:2026-07-21 13:43:59.228113675 +0000 UTC m=+0.379954795 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102806 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f478525f81566ea121a521d08ffba SystemUUID:ec2f4785-25f8-1566-ea12-1a521d08ffba BootID:be893f52-262f-437d-9701-e554752af73c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:37:29:03:8c:21 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:37:29:03:8c:21 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:8e:09:f6:bb:7c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 13:43:59.644877 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.229662 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 13:43:59.644877 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.229825 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 13:43:59.644877 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.233596 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 13:43:59.644877 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.233637 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-212.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 13:43:59.644877 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.234564 2567 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 13:43:59.644877 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.234591 2567 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 13:43:59.644877 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.234603 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.234624 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.236625 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.241487 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.241715 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.242510 2567 kubelet.go:486] "Attempting to sync node with API server" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.242531 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.242548 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.242566 2567 kubelet.go:398] "Adding apiserver pod source" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.242577 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.243175 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.243205 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.246177 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.246224 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.251468 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.251552 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.254063 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.254083 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.254089 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.254095 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.254104 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.254110 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.254119 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.254125 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.254132 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.254138 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.254146 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.254152 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.255966 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.255982 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.259922 2567 server.go:1268] "Started kubelet" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.259996 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.260062 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.260147 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.260442 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-212.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.260611 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.260612 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-212.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.262233 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.262688 2567 server.go:310] "Adding debug handlers to kubelet server" Jul 21 13:43:59.645287 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.265534 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.266897 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-212.ec2.internal.18c451eefe871e9f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-212.ec2.internal,UID:ip-10-0-129-212.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-212.ec2.internal,},FirstTimestamp:2026-07-21 13:43:59.259893407 +0000 UTC m=+0.411734506,LastTimestamp:2026-07-21 13:43:59.259893407 +0000 UTC m=+0.411734506,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-212.ec2.internal,}" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.271311 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.271317 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.271401 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.272094 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.272067 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.272117 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.273928 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.274722 2567 reconstruct.go:97] "Volume reconstruction finished" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.274731 2567 reconciler.go:29] "Reconciler: start to sync state" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.275434 2567 factory.go:55] Registering systemd factory Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.275450 2567 factory.go:223] Registration of the systemd container factory successfully Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.275637 2567 factory.go:153] Registering CRI-O factory Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.275645 2567 factory.go:223] Registration of the crio container factory successfully Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.275709 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.275731 2567 factory.go:103] Registering Raw factory Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.275743 2567 manager.go:1196] Started watching for new ooms in manager Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.276911 2567 manager.go:349] Starting recovery of all containers Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.285807 2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-212.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.287887 2567 manager.go:354] Recovery completed Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.295115 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.296913 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4rv9v" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.297232 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.297272 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.297283 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.297823 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.297832 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.297849 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.297849 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.298567 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.298592 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.298602 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Jul 21 13:43:59.646529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.300151 2567 policy_none.go:49] "None policy: Start" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.300166 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.300176 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.302132 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4rv9v" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.306629 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-212.ec2.internal.18c451ef00c12c54 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-212.ec2.internal,UID:ip-10-0-129-212.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-212.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-212.ec2.internal,},FirstTimestamp:2026-07-21 13:43:59.297252436 +0000 UTC m=+0.449093538,LastTimestamp:2026-07-21 13:43:59.297252436 +0000 UTC m=+0.449093538,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-212.ec2.internal,}" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.310949 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-212.ec2.internal.18c451ef00c18fd1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-212.ec2.internal,UID:ip-10-0-129-212.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-212.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-212.ec2.internal,},FirstTimestamp:2026-07-21 13:43:59.297277905 +0000 UTC m=+0.449119005,LastTimestamp:2026-07-21 13:43:59.297277905 +0000 UTC m=+0.449119005,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-212.ec2.internal,}" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.311941 2567 policy_none.go:47] "Start" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.372354 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.376544 2567 manager.go:337] "Starting Device Plugin manager" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.376592 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.376605 2567 server.go:87] "Starting device plugin registration server" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.377013 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.377028 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.377133 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.377257 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.377267 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.377828 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.377883 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.393127 2567 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2601 into the system container \"/system.slice\": " Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.477718 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.478839 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.478873 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.478886 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.478914 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.490060 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.647570 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.490085 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-212.ec2.internal\": node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.523812 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.524548 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.525787 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.525825 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.525850 2567 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.525886 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.529122 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.624511 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.626659 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal"] Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.626756 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.629830 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.629862 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.629872 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.632278 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.632379 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.632411 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.633235 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.633240 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.633259 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.633264 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.633269 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.633283 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.635566 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.635598 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.636377 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.636406 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 13:43:59.648308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.636420 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Jul 21 13:43:59.752545 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.676967 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/093d700f61b33cea8abfc261e9b28005-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal\" (UID: \"093d700f61b33cea8abfc261e9b28005\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.752545 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.676996 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/093d700f61b33cea8abfc261e9b28005-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal\" (UID: \"093d700f61b33cea8abfc261e9b28005\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.752545 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.677024 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ccec470417826b6e67569d0c3ca8973-config\") pod \"kube-apiserver-proxy-ip-10-0-129-212.ec2.internal\" (UID: \"6ccec470417826b6e67569d0c3ca8973\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.752545 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.706025 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-212.ec2.internal\" not found" node="ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.752545 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.710908 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-212.ec2.internal\" not found" node="ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.752545 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.725411 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:43:59.905547 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.777245 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/093d700f61b33cea8abfc261e9b28005-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal\" (UID: \"093d700f61b33cea8abfc261e9b28005\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.905547 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.777273 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/093d700f61b33cea8abfc261e9b28005-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal\" (UID: \"093d700f61b33cea8abfc261e9b28005\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.905547 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.777291 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ccec470417826b6e67569d0c3ca8973-config\") pod \"kube-apiserver-proxy-ip-10-0-129-212.ec2.internal\" (UID: \"6ccec470417826b6e67569d0c3ca8973\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.905547 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.777352 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/093d700f61b33cea8abfc261e9b28005-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal\" (UID: \"093d700f61b33cea8abfc261e9b28005\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.905547 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.777364 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/093d700f61b33cea8abfc261e9b28005-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal\" (UID: \"093d700f61b33cea8abfc261e9b28005\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.905547 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:43:59.777345 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6ccec470417826b6e67569d0c3ca8973-config\") pod \"kube-apiserver-proxy-ip-10-0-129-212.ec2.internal\" (UID: \"6ccec470417826b6e67569d0c3ca8973\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" Jul 21 13:43:59.905547 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.826306 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:44:00.071755 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:43:59.926515 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:44:00.071755 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.011421 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Jul 21 13:44:00.071755 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.013772 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" Jul 21 13:44:00.071755 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:00.026635 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:44:00.259578 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:00.127473 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:44:00.259578 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.196154 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 13:44:00.259578 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.196321 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 13:44:00.259578 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:00.228864 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:44:00.404383 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.316511 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 13:38:59 +0000 UTC" deadline="2028-01-28 21:12:12.105453911 +0000 UTC" Jul 21 13:44:00.404383 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.316559 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13351h28m11.78890358s" Jul 21 13:44:00.404383 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:00.329607 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:44:00.404383 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.341627 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 13:44:00.404383 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.342854 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientMemory" Jul 21 13:44:00.404383 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.342900 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 13:44:00.404383 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.342945 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeHasSufficientPID" Jul 21 13:44:00.577455 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:00.429895 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:44:00.577455 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.436008 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 13:44:00.577455 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.458578 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 13:44:00.577455 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:00.530086 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:44:00.699269 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:00.631248 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-212.ec2.internal\" not found" Jul 21 13:44:00.699269 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.690427 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 13:44:00.820788 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.772130 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" Jul 21 13:44:00.820788 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.787544 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 13:44:00.820788 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.788728 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" Jul 21 13:44:00.820788 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.797524 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 13:44:00.970729 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:00.848898 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod093d700f61b33cea8abfc261e9b28005.slice/crio-56064c4a6c02d619ac8b06736d3a94e66e547d8f51614daeb7715aab03323f9f WatchSource:0}: Error finding container 56064c4a6c02d619ac8b06736d3a94e66e547d8f51614daeb7715aab03323f9f: Status 404 returned error can't find the container with id 56064c4a6c02d619ac8b06736d3a94e66e547d8f51614daeb7715aab03323f9f Jul 21 13:44:00.970729 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:00.849128 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ccec470417826b6e67569d0c3ca8973.slice/crio-46147dfeb98e580e2283e65aef9c0dc94cbe87b2e4329a44e7fbb6e7ea1394d0 WatchSource:0}: Error finding container 46147dfeb98e580e2283e65aef9c0dc94cbe87b2e4329a44e7fbb6e7ea1394d0: Status 404 returned error can't find the container with id 46147dfeb98e580e2283e65aef9c0dc94cbe87b2e4329a44e7fbb6e7ea1394d0 Jul 21 13:44:00.970729 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:00.854175 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.243611 2567 apiserver.go:52] "Watching apiserver" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.256205 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.257550 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-5vvn4","kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4","openshift-dns/node-resolver-js4ds","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal","openshift-multus/multus-5zc66","openshift-multus/multus-additional-cni-plugins-t26mf","openshift-network-diagnostics/network-check-target-2cps2","kube-system/global-pull-secret-syncer-xvrvl","openshift-cluster-node-tuning-operator/tuned-wccx6","openshift-image-registry/node-ca-bbnk6","openshift-multus/network-metrics-daemon-xlpnd","openshift-network-operator/iptables-alerter-9hn22","openshift-ovn-kubernetes/ovnkube-node-h66bv"] Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.261512 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:01.261581 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.263877 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.266370 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9hn22" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.268380 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.268530 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.268869 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xkh97\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.269030 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bbnk6" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.270082 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.270288 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.270668 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wlqmd\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.270709 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.270833 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.270963 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.271081 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.271358 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.272136 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.272486 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.273906 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.276483 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5vvn4" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.277506 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.277543 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.277554 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 13:44:01.295701 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.277894 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-77mnp\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.277902 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.277921 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.277893 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.277924 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.278281 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.278332 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l7495\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.278513 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.278514 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-255sv\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.279758 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:01.279853 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.280230 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hxvbv\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.280314 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.281081 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.283864 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.286484 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-js4ds" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.286983 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95b64ba6-e701-4efb-bc64-74565efa56e7-cni-binary-copy\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.287019 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-var-lib-openvswitch\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.287049 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-run-ovn\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.287073 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c694404b-a00c-435e-8d91-695f127abbb0-ovnkube-config\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.287134 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c694404b-a00c-435e-8d91-695f127abbb0-ovnkube-script-lib\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.287611 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h8zq8\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.287928 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.287977 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 13:44:01.298456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.289204 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.289357 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gjlmz\"" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.291262 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07055c88-a7fe-4387-b48f-b4eed49ed54d-iptables-alerter-script\") pod \"iptables-alerter-9hn22\" (UID: \"07055c88-a7fe-4387-b48f-b4eed49ed54d\") " pod="openshift-network-operator/iptables-alerter-9hn22" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.291287 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.291357 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2a5ace1-fdd3-4841-992a-a0f34c621d83-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.291393 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c2a5ace1-fdd3-4841-992a-a0f34c621d83-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.291423 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-multus-cni-dir\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.291457 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-cni-bin\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.291522 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c694404b-a00c-435e-8d91-695f127abbb0-ovn-node-metrics-cert\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.291970 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2a5ace1-fdd3-4841-992a-a0f34c621d83-os-release\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.291998 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-os-release\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292059 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-var-lib-cni-multus\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292097 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-run-multus-certs\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292134 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szx26\" (UniqueName: \"kubernetes.io/projected/95b64ba6-e701-4efb-bc64-74565efa56e7-kube-api-access-szx26\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.299382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292155 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292173 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2a5ace1-fdd3-4841-992a-a0f34c621d83-cnibin\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292190 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2a5ace1-fdd3-4841-992a-a0f34c621d83-cni-binary-copy\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292221 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c2a5ace1-fdd3-4841-992a-a0f34c621d83-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292276 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-system-cni-dir\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292309 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-etc-kubernetes\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292336 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-run-systemd\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292355 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-node-log\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292375 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-log-socket\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292396 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f26b564f-04fd-4e02-ade5-45adcf879547-host\") pod \"node-ca-bbnk6\" (UID: \"f26b564f-04fd-4e02-ade5-45adcf879547\") " pod="openshift-image-registry/node-ca-bbnk6" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292423 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66vwm\" (UniqueName: \"kubernetes.io/projected/c2a5ace1-fdd3-4841-992a-a0f34c621d83-kube-api-access-66vwm\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292440 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-multus-socket-dir-parent\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292457 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-hostroot\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292479 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95b64ba6-e701-4efb-bc64-74565efa56e7-multus-daemon-config\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.300664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292505 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07055c88-a7fe-4387-b48f-b4eed49ed54d-host-slash\") pod \"iptables-alerter-9hn22\" (UID: \"07055c88-a7fe-4387-b48f-b4eed49ed54d\") " pod="openshift-network-operator/iptables-alerter-9hn22" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292521 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-cnibin\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292535 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-var-lib-kubelet\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292549 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-multus-conf-dir\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292571 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhhff\" (UniqueName: \"kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff\") pod \"network-check-target-2cps2\" (UID: \"4cfe6a65-9649-46d6-a3c2-723dc4e9e218\") " pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292594 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-systemd-units\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292613 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-run-netns\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292640 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-etc-openvswitch\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292665 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c694404b-a00c-435e-8d91-695f127abbb0-env-overrides\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292691 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qrqj\" (UniqueName: \"kubernetes.io/projected/07055c88-a7fe-4387-b48f-b4eed49ed54d-kube-api-access-7qrqj\") pod \"iptables-alerter-9hn22\" (UID: \"07055c88-a7fe-4387-b48f-b4eed49ed54d\") " pod="openshift-network-operator/iptables-alerter-9hn22" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292757 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb47n\" (UniqueName: \"kubernetes.io/projected/f26b564f-04fd-4e02-ade5-45adcf879547-kube-api-access-sb47n\") pod \"node-ca-bbnk6\" (UID: \"f26b564f-04fd-4e02-ade5-45adcf879547\") " pod="openshift-image-registry/node-ca-bbnk6" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292813 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2a5ace1-fdd3-4841-992a-a0f34c621d83-system-cni-dir\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292838 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-slash\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292863 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-run-openvswitch\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.301351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292888 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-cni-netd\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292909 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cvzs\" (UniqueName: \"kubernetes.io/projected/c694404b-a00c-435e-8d91-695f127abbb0-kube-api-access-7cvzs\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292933 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f26b564f-04fd-4e02-ade5-45adcf879547-serviceca\") pod \"node-ca-bbnk6\" (UID: \"f26b564f-04fd-4e02-ade5-45adcf879547\") " pod="openshift-image-registry/node-ca-bbnk6" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292958 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-run-k8s-cni-cncf-io\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.292978 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-run-netns\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.293002 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-var-lib-cni-bin\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.293038 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-kubelet\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.293066 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-run-ovn-kubernetes\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.294352 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.294726 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:01.294794 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.297040 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.301217 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.301374 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.301508 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 13:44:01.303119 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.301533 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xh5b9\"" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.339888 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bbxrc" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.350311 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bbxrc" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.372999 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395189 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c694404b-a00c-435e-8d91-695f127abbb0-ovn-node-metrics-cert\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395232 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2a5ace1-fdd3-4841-992a-a0f34c621d83-os-release\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395259 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs\") pod \"network-metrics-daemon-xlpnd\" (UID: \"d08982d1-044f-462c-a84a-d4c65f1cbbef\") " pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395278 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-var-lib-kubelet\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395297 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-socket-dir\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395324 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2a5ace1-fdd3-4841-992a-a0f34c621d83-cnibin\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395350 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c2a5ace1-fdd3-4841-992a-a0f34c621d83-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395407 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-system-cni-dir\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395409 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2a5ace1-fdd3-4841-992a-a0f34c621d83-os-release\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395442 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2a5ace1-fdd3-4841-992a-a0f34c621d83-cnibin\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395492 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-etc-kubernetes\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395525 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-system-cni-dir\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.416939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395557 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-node-log\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395549 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395583 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-log-socket\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395602 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f26b564f-04fd-4e02-ade5-45adcf879547-host\") pod \"node-ca-bbnk6\" (UID: \"f26b564f-04fd-4e02-ade5-45adcf879547\") " pod="openshift-image-registry/node-ca-bbnk6" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395617 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-etc-kubernetes\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395632 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66vwm\" (UniqueName: \"kubernetes.io/projected/c2a5ace1-fdd3-4841-992a-a0f34c621d83-kube-api-access-66vwm\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395657 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-node-log\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395664 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-multus-socket-dir-parent\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395674 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f26b564f-04fd-4e02-ade5-45adcf879547-host\") pod \"node-ca-bbnk6\" (UID: \"f26b564f-04fd-4e02-ade5-45adcf879547\") " pod="openshift-image-registry/node-ca-bbnk6" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395742 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-log-socket\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395772 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-hostroot\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395815 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-kubelet-config\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395843 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-sysconfig\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395862 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-multus-socket-dir-parent\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395894 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-hostroot\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.419041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395905 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07055c88-a7fe-4387-b48f-b4eed49ed54d-host-slash\") pod \"iptables-alerter-9hn22\" (UID: \"07055c88-a7fe-4387-b48f-b4eed49ed54d\") " pod="openshift-network-operator/iptables-alerter-9hn22" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395936 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhhff\" (UniqueName: \"kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff\") pod \"network-check-target-2cps2\" (UID: \"4cfe6a65-9649-46d6-a3c2-723dc4e9e218\") " pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395969 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c2a5ace1-fdd3-4841-992a-a0f34c621d83-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395978 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/07055c88-a7fe-4387-b48f-b4eed49ed54d-host-slash\") pod \"iptables-alerter-9hn22\" (UID: \"07055c88-a7fe-4387-b48f-b4eed49ed54d\") " pod="openshift-network-operator/iptables-alerter-9hn22" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.395982 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-sysctl-d\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396098 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-sysctl-conf\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396126 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-host\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396156 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb47n\" (UniqueName: \"kubernetes.io/projected/f26b564f-04fd-4e02-ade5-45adcf879547-kube-api-access-sb47n\") pod \"node-ca-bbnk6\" (UID: \"f26b564f-04fd-4e02-ade5-45adcf879547\") " pod="openshift-image-registry/node-ca-bbnk6" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396189 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2a5ace1-fdd3-4841-992a-a0f34c621d83-system-cni-dir\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396221 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-run-netns\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396257 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-var-lib-cni-bin\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396266 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2a5ace1-fdd3-4841-992a-a0f34c621d83-system-cni-dir\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396285 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-systemd\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396312 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-run-netns\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.419687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396311 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-registration-dir\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396344 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-device-dir\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396359 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-var-lib-cni-bin\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396373 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-sys-fs\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396395 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-slash\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396417 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cvzs\" (UniqueName: \"kubernetes.io/projected/c694404b-a00c-435e-8d91-695f127abbb0-kube-api-access-7cvzs\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396438 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67843201-85e5-47e7-8100-c1bdecbefcfe-konnectivity-ca\") pod \"konnectivity-agent-5vvn4\" (UID: \"67843201-85e5-47e7-8100-c1bdecbefcfe\") " pod="kube-system/konnectivity-agent-5vvn4" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396459 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-slash\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396465 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvfhd\" (UniqueName: \"kubernetes.io/projected/554c4657-e117-4fbb-8b28-a71edec5e054-kube-api-access-qvfhd\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396494 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b19a38a-ffc0-4c33-914c-0eb1b2c67b71-hosts-file\") pod \"node-resolver-js4ds\" (UID: \"2b19a38a-ffc0-4c33-914c-0eb1b2c67b71\") " pod="openshift-dns/node-resolver-js4ds" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396512 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-kubelet\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396528 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95b64ba6-e701-4efb-bc64-74565efa56e7-cni-binary-copy\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396544 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396563 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mp58\" (UniqueName: \"kubernetes.io/projected/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-kube-api-access-6mp58\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.420564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396581 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c694404b-a00c-435e-8d91-695f127abbb0-ovnkube-script-lib\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396598 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07055c88-a7fe-4387-b48f-b4eed49ed54d-iptables-alerter-script\") pod \"iptables-alerter-9hn22\" (UID: \"07055c88-a7fe-4387-b48f-b4eed49ed54d\") " pod="openshift-network-operator/iptables-alerter-9hn22" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396615 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c2a5ace1-fdd3-4841-992a-a0f34c621d83-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397042 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c2a5ace1-fdd3-4841-992a-a0f34c621d83-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.396595 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-kubelet\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397109 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-run-multus-certs\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397142 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szx26\" (UniqueName: \"kubernetes.io/projected/95b64ba6-e701-4efb-bc64-74565efa56e7-kube-api-access-szx26\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397179 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-lib-modules\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397207 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-cni-bin\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397219 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/07055c88-a7fe-4387-b48f-b4eed49ed54d-iptables-alerter-script\") pod \"iptables-alerter-9hn22\" (UID: \"07055c88-a7fe-4387-b48f-b4eed49ed54d\") " pod="openshift-network-operator/iptables-alerter-9hn22" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397230 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-os-release\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397232 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95b64ba6-e701-4efb-bc64-74565efa56e7-cni-binary-copy\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397253 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-var-lib-cni-multus\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397344 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-os-release\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.421199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397366 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-dbus\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397392 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-cni-bin\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397393 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-run\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397432 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397438 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c694404b-a00c-435e-8d91-695f127abbb0-ovnkube-script-lib\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397438 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-run-multus-certs\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397469 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2a5ace1-fdd3-4841-992a-a0f34c621d83-cni-binary-copy\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397481 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397488 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-modprobe-d\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397508 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-run-systemd\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397509 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-var-lib-cni-multus\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397524 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95b64ba6-e701-4efb-bc64-74565efa56e7-multus-daemon-config\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397550 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-sys\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397555 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-run-systemd\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.422036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397570 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-cnibin\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397586 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-var-lib-kubelet\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397625 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-var-lib-kubelet\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397645 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-multus-conf-dir\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397662 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67843201-85e5-47e7-8100-c1bdecbefcfe-agent-certs\") pod \"konnectivity-agent-5vvn4\" (UID: \"67843201-85e5-47e7-8100-c1bdecbefcfe\") " pod="kube-system/konnectivity-agent-5vvn4" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397676 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-kubernetes\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397693 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397711 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-systemd-units\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397696 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-cnibin\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397751 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-run-netns\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397770 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-etc-openvswitch\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397801 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c694404b-a00c-435e-8d91-695f127abbb0-env-overrides\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397839 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-run-netns\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397872 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qrqj\" (UniqueName: \"kubernetes.io/projected/07055c88-a7fe-4387-b48f-b4eed49ed54d-kube-api-access-7qrqj\") pod \"iptables-alerter-9hn22\" (UID: \"07055c88-a7fe-4387-b48f-b4eed49ed54d\") " pod="openshift-network-operator/iptables-alerter-9hn22" Jul 21 13:44:01.422531 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397903 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-run-k8s-cni-cncf-io\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397931 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b19a38a-ffc0-4c33-914c-0eb1b2c67b71-tmp-dir\") pod \"node-resolver-js4ds\" (UID: \"2b19a38a-ffc0-4c33-914c-0eb1b2c67b71\") " pod="openshift-dns/node-resolver-js4ds" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397957 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-run-openvswitch\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397982 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-cni-netd\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398009 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f26b564f-04fd-4e02-ade5-45adcf879547-serviceca\") pod \"node-ca-bbnk6\" (UID: \"f26b564f-04fd-4e02-ade5-45adcf879547\") " pod="openshift-image-registry/node-ca-bbnk6" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398022 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95b64ba6-e701-4efb-bc64-74565efa56e7-multus-daemon-config\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398045 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjjbn\" (UniqueName: \"kubernetes.io/projected/d08982d1-044f-462c-a84a-d4c65f1cbbef-kube-api-access-cjjbn\") pod \"network-metrics-daemon-xlpnd\" (UID: \"d08982d1-044f-462c-a84a-d4c65f1cbbef\") " pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398070 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c694404b-a00c-435e-8d91-695f127abbb0-env-overrides\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398075 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmjbr\" (UniqueName: \"kubernetes.io/projected/2b19a38a-ffc0-4c33-914c-0eb1b2c67b71-kube-api-access-jmjbr\") pod \"node-resolver-js4ds\" (UID: \"2b19a38a-ffc0-4c33-914c-0eb1b2c67b71\") " pod="openshift-dns/node-resolver-js4ds" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398080 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-systemd-units\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.397779 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-multus-conf-dir\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398123 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-etc-openvswitch\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398136 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-cni-netd\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398009 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2a5ace1-fdd3-4841-992a-a0f34c621d83-cni-binary-copy\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398146 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-run-openvswitch\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398047 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-host-run-k8s-cni-cncf-io\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398151 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-etc-selinux\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398175 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-run-ovn-kubernetes\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398192 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/554c4657-e117-4fbb-8b28-a71edec5e054-etc-tuned\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398209 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/554c4657-e117-4fbb-8b28-a71edec5e054-tmp\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398252 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-var-lib-openvswitch\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398268 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-run-ovn\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398287 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c694404b-a00c-435e-8d91-695f127abbb0-ovnkube-config\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398287 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-host-run-ovn-kubernetes\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398305 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2a5ace1-fdd3-4841-992a-a0f34c621d83-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398326 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-multus-cni-dir\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398332 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-var-lib-openvswitch\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398336 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c694404b-a00c-435e-8d91-695f127abbb0-run-ovn\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398468 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95b64ba6-e701-4efb-bc64-74565efa56e7-multus-cni-dir\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.423756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398532 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f26b564f-04fd-4e02-ade5-45adcf879547-serviceca\") pod \"node-ca-bbnk6\" (UID: \"f26b564f-04fd-4e02-ade5-45adcf879547\") " pod="openshift-image-registry/node-ca-bbnk6" Jul 21 13:44:01.424431 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398557 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2a5ace1-fdd3-4841-992a-a0f34c621d83-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.424431 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.398903 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c694404b-a00c-435e-8d91-695f127abbb0-ovnkube-config\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.424431 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.399773 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c694404b-a00c-435e-8d91-695f127abbb0-ovn-node-metrics-cert\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.424431 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:01.408645 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 13:44:01.424431 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:01.408663 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 13:44:01.424431 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:01.408673 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xhhff for pod openshift-network-diagnostics/network-check-target-2cps2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:01.424431 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:01.408750 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff podName:4cfe6a65-9649-46d6-a3c2-723dc4e9e218 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:01.908731972 +0000 UTC m=+3.060573075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xhhff" (UniqueName: "kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff") pod "network-check-target-2cps2" (UID: "4cfe6a65-9649-46d6-a3c2-723dc4e9e218") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:01.424431 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.409373 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66vwm\" (UniqueName: \"kubernetes.io/projected/c2a5ace1-fdd3-4841-992a-a0f34c621d83-kube-api-access-66vwm\") pod \"multus-additional-cni-plugins-t26mf\" (UID: \"c2a5ace1-fdd3-4841-992a-a0f34c621d83\") " pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.424431 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.412037 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qrqj\" (UniqueName: \"kubernetes.io/projected/07055c88-a7fe-4387-b48f-b4eed49ed54d-kube-api-access-7qrqj\") pod \"iptables-alerter-9hn22\" (UID: \"07055c88-a7fe-4387-b48f-b4eed49ed54d\") " pod="openshift-network-operator/iptables-alerter-9hn22" Jul 21 13:44:01.424431 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.412581 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szx26\" (UniqueName: \"kubernetes.io/projected/95b64ba6-e701-4efb-bc64-74565efa56e7-kube-api-access-szx26\") pod \"multus-5zc66\" (UID: \"95b64ba6-e701-4efb-bc64-74565efa56e7\") " pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.424431 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.413075 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb47n\" (UniqueName: \"kubernetes.io/projected/f26b564f-04fd-4e02-ade5-45adcf879547-kube-api-access-sb47n\") pod \"node-ca-bbnk6\" (UID: \"f26b564f-04fd-4e02-ade5-45adcf879547\") " pod="openshift-image-registry/node-ca-bbnk6" Jul 21 13:44:01.424431 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.420573 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cvzs\" (UniqueName: \"kubernetes.io/projected/c694404b-a00c-435e-8d91-695f127abbb0-kube-api-access-7cvzs\") pod \"ovnkube-node-h66bv\" (UID: \"c694404b-a00c-435e-8d91-695f127abbb0\") " pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.433525 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.498911 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67843201-85e5-47e7-8100-c1bdecbefcfe-konnectivity-ca\") pod \"konnectivity-agent-5vvn4\" (UID: \"67843201-85e5-47e7-8100-c1bdecbefcfe\") " pod="kube-system/konnectivity-agent-5vvn4" Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.498959 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvfhd\" (UniqueName: \"kubernetes.io/projected/554c4657-e117-4fbb-8b28-a71edec5e054-kube-api-access-qvfhd\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.498987 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b19a38a-ffc0-4c33-914c-0eb1b2c67b71-hosts-file\") pod \"node-resolver-js4ds\" (UID: \"2b19a38a-ffc0-4c33-914c-0eb1b2c67b71\") " pod="openshift-dns/node-resolver-js4ds" Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499019 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499047 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mp58\" (UniqueName: \"kubernetes.io/projected/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-kube-api-access-6mp58\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499076 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-lib-modules\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499101 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-dbus\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499100 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2b19a38a-ffc0-4c33-914c-0eb1b2c67b71-hosts-file\") pod \"node-resolver-js4ds\" (UID: \"2b19a38a-ffc0-4c33-914c-0eb1b2c67b71\") " pod="openshift-dns/node-resolver-js4ds" Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499127 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-run\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499154 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-modprobe-d\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:01.499178 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499228 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-sys\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:01.499262 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret podName:093b3a4b-ec79-4a4d-a86f-05ea62a88e2c nodeName:}" failed. No retries permitted until 2026-07-21 13:44:01.999241331 +0000 UTC m=+3.151082441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret") pod "global-pull-secret-syncer-xvrvl" (UID: "093b3a4b-ec79-4a4d-a86f-05ea62a88e2c") : object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:01.548610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499178 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-sys\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499301 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-run\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499307 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67843201-85e5-47e7-8100-c1bdecbefcfe-agent-certs\") pod \"konnectivity-agent-5vvn4\" (UID: \"67843201-85e5-47e7-8100-c1bdecbefcfe\") " pod="kube-system/konnectivity-agent-5vvn4" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499340 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-modprobe-d\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499358 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-kubernetes\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499386 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499394 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-dbus\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499427 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b19a38a-ffc0-4c33-914c-0eb1b2c67b71-tmp-dir\") pod \"node-resolver-js4ds\" (UID: \"2b19a38a-ffc0-4c33-914c-0eb1b2c67b71\") " pod="openshift-dns/node-resolver-js4ds" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499451 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499458 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjjbn\" (UniqueName: \"kubernetes.io/projected/d08982d1-044f-462c-a84a-d4c65f1cbbef-kube-api-access-cjjbn\") pod \"network-metrics-daemon-xlpnd\" (UID: \"d08982d1-044f-462c-a84a-d4c65f1cbbef\") " pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499485 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmjbr\" (UniqueName: \"kubernetes.io/projected/2b19a38a-ffc0-4c33-914c-0eb1b2c67b71-kube-api-access-jmjbr\") pod \"node-resolver-js4ds\" (UID: \"2b19a38a-ffc0-4c33-914c-0eb1b2c67b71\") " pod="openshift-dns/node-resolver-js4ds" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499501 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-kubernetes\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499510 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-etc-selinux\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499535 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/554c4657-e117-4fbb-8b28-a71edec5e054-etc-tuned\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499558 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/554c4657-e117-4fbb-8b28-a71edec5e054-tmp\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.550484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499590 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs\") pod \"network-metrics-daemon-xlpnd\" (UID: \"d08982d1-044f-462c-a84a-d4c65f1cbbef\") " pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499595 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67843201-85e5-47e7-8100-c1bdecbefcfe-konnectivity-ca\") pod \"konnectivity-agent-5vvn4\" (UID: \"67843201-85e5-47e7-8100-c1bdecbefcfe\") " pod="kube-system/konnectivity-agent-5vvn4" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499616 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-var-lib-kubelet\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499642 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-socket-dir\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499683 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-kubelet-config\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499259 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-lib-modules\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499710 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-sysconfig\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499750 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-sysctl-d\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499774 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-sysctl-conf\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499815 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-host\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499844 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-systemd\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499873 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-registration-dir\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499896 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b19a38a-ffc0-4c33-914c-0eb1b2c67b71-tmp-dir\") pod \"node-resolver-js4ds\" (UID: \"2b19a38a-ffc0-4c33-914c-0eb1b2c67b71\") " pod="openshift-dns/node-resolver-js4ds" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499897 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-device-dir\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.551586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499902 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-etc-selinux\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499946 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-device-dir\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499946 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-sys-fs\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.499994 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-sys-fs\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:01.500056 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:01.500114 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs podName:d08982d1-044f-462c-a84a-d4c65f1cbbef nodeName:}" failed. No retries permitted until 2026-07-21 13:44:02.000098079 +0000 UTC m=+3.151939166 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs") pod "network-metrics-daemon-xlpnd" (UID: "d08982d1-044f-462c-a84a-d4c65f1cbbef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.500121 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-sysctl-d\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.500153 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-socket-dir\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.500167 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-kubelet-config\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.500224 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-sysconfig\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.500229 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-var-lib-kubelet\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.500241 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-sysctl-conf\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.500270 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-host\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.500304 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-registration-dir\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.554704 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.500313 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/554c4657-e117-4fbb-8b28-a71edec5e054-etc-systemd\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.555572 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.501937 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/554c4657-e117-4fbb-8b28-a71edec5e054-tmp\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.555572 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.502052 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67843201-85e5-47e7-8100-c1bdecbefcfe-agent-certs\") pod \"konnectivity-agent-5vvn4\" (UID: \"67843201-85e5-47e7-8100-c1bdecbefcfe\") " pod="kube-system/konnectivity-agent-5vvn4" Jul 21 13:44:01.555572 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.502489 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/554c4657-e117-4fbb-8b28-a71edec5e054-etc-tuned\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.555572 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.518810 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmjbr\" (UniqueName: \"kubernetes.io/projected/2b19a38a-ffc0-4c33-914c-0eb1b2c67b71-kube-api-access-jmjbr\") pod \"node-resolver-js4ds\" (UID: \"2b19a38a-ffc0-4c33-914c-0eb1b2c67b71\") " pod="openshift-dns/node-resolver-js4ds" Jul 21 13:44:01.555572 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.519418 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvfhd\" (UniqueName: \"kubernetes.io/projected/554c4657-e117-4fbb-8b28-a71edec5e054-kube-api-access-qvfhd\") pod \"tuned-wccx6\" (UID: \"554c4657-e117-4fbb-8b28-a71edec5e054\") " pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.555572 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.520151 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mp58\" (UniqueName: \"kubernetes.io/projected/d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3-kube-api-access-6mp58\") pod \"aws-ebs-csi-driver-node-fpnw4\" (UID: \"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.555572 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.524696 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjjbn\" (UniqueName: \"kubernetes.io/projected/d08982d1-044f-462c-a84a-d4c65f1cbbef-kube-api-access-cjjbn\") pod \"network-metrics-daemon-xlpnd\" (UID: \"d08982d1-044f-462c-a84a-d4c65f1cbbef\") " pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:01.555572 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.536448 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" event={"ID":"6ccec470417826b6e67569d0c3ca8973","Type":"ContainerStarted","Data":"46147dfeb98e580e2283e65aef9c0dc94cbe87b2e4329a44e7fbb6e7ea1394d0"} Jul 21 13:44:01.555572 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.537646 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" event={"ID":"093d700f61b33cea8abfc261e9b28005","Type":"ContainerStarted","Data":"56064c4a6c02d619ac8b06736d3a94e66e547d8f51614daeb7715aab03323f9f"} Jul 21 13:44:01.626127 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.575488 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:01.626127 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.592272 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9hn22" Jul 21 13:44:01.626127 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:01.599567 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc694404b_a00c_435e_8d91_695f127abbb0.slice/crio-84d6db1129e91d91a91479468c1cd6979d8f370570a504f69dbe562a5ed515a5 WatchSource:0}: Error finding container 84d6db1129e91d91a91479468c1cd6979d8f370570a504f69dbe562a5ed515a5: Status 404 returned error can't find the container with id 84d6db1129e91d91a91479468c1cd6979d8f370570a504f69dbe562a5ed515a5 Jul 21 13:44:01.626127 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.601529 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bbnk6" Jul 21 13:44:01.626127 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:01.604442 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07055c88_a7fe_4387_b48f_b4eed49ed54d.slice/crio-58484a189b5d22816d2dee2ea984ba4fe501a8d0f545e89069619d98d6eab97c WatchSource:0}: Error finding container 58484a189b5d22816d2dee2ea984ba4fe501a8d0f545e89069619d98d6eab97c: Status 404 returned error can't find the container with id 58484a189b5d22816d2dee2ea984ba4fe501a8d0f545e89069619d98d6eab97c Jul 21 13:44:01.626127 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.606875 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t26mf" Jul 21 13:44:01.626127 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.613989 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5zc66" Jul 21 13:44:01.626127 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.622341 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5vvn4" Jul 21 13:44:01.712761 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.628465 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wccx6" Jul 21 13:44:01.712761 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.640630 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-js4ds" Jul 21 13:44:01.712761 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:01.645931 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" Jul 21 13:44:01.712761 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:01.670634 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67843201_85e5_47e7_8100_c1bdecbefcfe.slice/crio-5703100d4a07e6b5e2330cb78db58b59da4b8a0e3bd4595e89c29672afe63d4d WatchSource:0}: Error finding container 5703100d4a07e6b5e2330cb78db58b59da4b8a0e3bd4595e89c29672afe63d4d: Status 404 returned error can't find the container with id 5703100d4a07e6b5e2330cb78db58b59da4b8a0e3bd4595e89c29672afe63d4d Jul 21 13:44:01.712761 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:01.671317 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b64ba6_e701_4efb_bc64_74565efa56e7.slice/crio-aaf66f938a574a293b5e54d481e89dc2407b8e17ffd1c6484c2864de7b493b1b WatchSource:0}: Error finding container aaf66f938a574a293b5e54d481e89dc2407b8e17ffd1c6484c2864de7b493b1b: Status 404 returned error can't find the container with id aaf66f938a574a293b5e54d481e89dc2407b8e17ffd1c6484c2864de7b493b1b Jul 21 13:44:01.712761 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:01.673123 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod554c4657_e117_4fbb_8b28_a71edec5e054.slice/crio-797873bd5a9db17dba4566008885d74d0a32a30407753d2a9b574ed56130e643 WatchSource:0}: Error finding container 797873bd5a9db17dba4566008885d74d0a32a30407753d2a9b574ed56130e643: Status 404 returned error can't find the container with id 797873bd5a9db17dba4566008885d74d0a32a30407753d2a9b574ed56130e643 Jul 21 13:44:01.712761 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:01.676375 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd434a8ee_ed0c_41a7_bdc9_b804f43c6dd3.slice/crio-a7ea10790f2a2413eb515cc07b16dc4c8f4286b411b9a0966f51fbe2087b53f7 WatchSource:0}: Error finding container a7ea10790f2a2413eb515cc07b16dc4c8f4286b411b9a0966f51fbe2087b53f7: Status 404 returned error can't find the container with id a7ea10790f2a2413eb515cc07b16dc4c8f4286b411b9a0966f51fbe2087b53f7 Jul 21 13:44:02.005144 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.005037 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:02.005144 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.005137 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs\") pod \"network-metrics-daemon-xlpnd\" (UID: \"d08982d1-044f-462c-a84a-d4c65f1cbbef\") " pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:02.005404 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.005181 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhhff\" (UniqueName: \"kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff\") pod \"network-check-target-2cps2\" (UID: \"4cfe6a65-9649-46d6-a3c2-723dc4e9e218\") " pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:02.005404 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:02.005206 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:02.005404 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:02.005292 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret podName:093b3a4b-ec79-4a4d-a86f-05ea62a88e2c nodeName:}" failed. No retries permitted until 2026-07-21 13:44:03.005268571 +0000 UTC m=+4.157109675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret") pod "global-pull-secret-syncer-xvrvl" (UID: "093b3a4b-ec79-4a4d-a86f-05ea62a88e2c") : object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:02.005404 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:02.005298 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:02.005404 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:02.005317 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 13:44:02.005404 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:02.005333 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 13:44:02.005404 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:02.005349 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xhhff for pod openshift-network-diagnostics/network-check-target-2cps2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:02.005404 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:02.005364 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs podName:d08982d1-044f-462c-a84a-d4c65f1cbbef nodeName:}" failed. No retries permitted until 2026-07-21 13:44:03.005344919 +0000 UTC m=+4.157186025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs") pod "network-metrics-daemon-xlpnd" (UID: "d08982d1-044f-462c-a84a-d4c65f1cbbef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:02.005404 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:02.005396 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff podName:4cfe6a65-9649-46d6-a3c2-723dc4e9e218 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:03.005381824 +0000 UTC m=+4.157222928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhhff" (UniqueName: "kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff") pod "network-check-target-2cps2" (UID: "4cfe6a65-9649-46d6-a3c2-723dc4e9e218") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:02.354013 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.353892 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 13:39:01 +0000 UTC" deadline="2028-04-09 10:08:45.217122748 +0000 UTC" Jul 21 13:44:02.354013 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.353948 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15068h24m42.863178631s" Jul 21 13:44:02.526488 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.526453 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:02.526703 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:02.526679 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:02.541424 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.541376 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" event={"ID":"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3","Type":"ContainerStarted","Data":"a7ea10790f2a2413eb515cc07b16dc4c8f4286b411b9a0966f51fbe2087b53f7"} Jul 21 13:44:02.542699 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.542670 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-js4ds" event={"ID":"2b19a38a-ffc0-4c33-914c-0eb1b2c67b71","Type":"ContainerStarted","Data":"75841f42a3cb75249c067682a480ea2ddaf334041af65d59bef7a5a1b239280f"} Jul 21 13:44:02.543927 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.543899 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bbnk6" event={"ID":"f26b564f-04fd-4e02-ade5-45adcf879547","Type":"ContainerStarted","Data":"ddb26124d64a205a6ad320d41ee6e01b04456d422744214b824956539f37777f"} Jul 21 13:44:02.545164 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.545132 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wccx6" event={"ID":"554c4657-e117-4fbb-8b28-a71edec5e054","Type":"ContainerStarted","Data":"797873bd5a9db17dba4566008885d74d0a32a30407753d2a9b574ed56130e643"} Jul 21 13:44:02.546444 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.546418 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5vvn4" event={"ID":"67843201-85e5-47e7-8100-c1bdecbefcfe","Type":"ContainerStarted","Data":"5703100d4a07e6b5e2330cb78db58b59da4b8a0e3bd4595e89c29672afe63d4d"} Jul 21 13:44:02.547681 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.547653 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5zc66" event={"ID":"95b64ba6-e701-4efb-bc64-74565efa56e7","Type":"ContainerStarted","Data":"aaf66f938a574a293b5e54d481e89dc2407b8e17ffd1c6484c2864de7b493b1b"} Jul 21 13:44:02.548987 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.548960 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t26mf" event={"ID":"c2a5ace1-fdd3-4841-992a-a0f34c621d83","Type":"ContainerStarted","Data":"208ae44e0ebd315d7aede18b6ffd73f8c8d1c25e06af5f637a6b9c5e0ac7851f"} Jul 21 13:44:02.550275 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.550249 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9hn22" event={"ID":"07055c88-a7fe-4387-b48f-b4eed49ed54d","Type":"ContainerStarted","Data":"58484a189b5d22816d2dee2ea984ba4fe501a8d0f545e89069619d98d6eab97c"} Jul 21 13:44:02.551533 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:02.551510 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" event={"ID":"c694404b-a00c-435e-8d91-695f127abbb0","Type":"ContainerStarted","Data":"84d6db1129e91d91a91479468c1cd6979d8f370570a504f69dbe562a5ed515a5"} Jul 21 13:44:03.014544 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:03.014500 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs\") pod \"network-metrics-daemon-xlpnd\" (UID: \"d08982d1-044f-462c-a84a-d4c65f1cbbef\") " pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:03.014780 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:03.014573 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhhff\" (UniqueName: \"kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff\") pod \"network-check-target-2cps2\" (UID: \"4cfe6a65-9649-46d6-a3c2-723dc4e9e218\") " pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:03.014780 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:03.014619 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:03.014780 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:03.014692 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:03.014780 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:03.014733 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:03.014780 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:03.014775 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 13:44:03.015206 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:03.014791 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs podName:d08982d1-044f-462c-a84a-d4c65f1cbbef nodeName:}" failed. No retries permitted until 2026-07-21 13:44:05.014767579 +0000 UTC m=+6.166608691 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs") pod "network-metrics-daemon-xlpnd" (UID: "d08982d1-044f-462c-a84a-d4c65f1cbbef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:03.015206 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:03.014800 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 13:44:03.015206 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:03.014816 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xhhff for pod openshift-network-diagnostics/network-check-target-2cps2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:03.015206 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:03.014813 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret podName:093b3a4b-ec79-4a4d-a86f-05ea62a88e2c nodeName:}" failed. No retries permitted until 2026-07-21 13:44:05.014802273 +0000 UTC m=+6.166643368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret") pod "global-pull-secret-syncer-xvrvl" (UID: "093b3a4b-ec79-4a4d-a86f-05ea62a88e2c") : object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:03.015206 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:03.014870 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff podName:4cfe6a65-9649-46d6-a3c2-723dc4e9e218 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:05.014849962 +0000 UTC m=+6.166691066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhhff" (UniqueName: "kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff") pod "network-check-target-2cps2" (UID: "4cfe6a65-9649-46d6-a3c2-723dc4e9e218") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:03.354384 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:03.354331 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 13:39:01 +0000 UTC" deadline="2028-03-19 02:35:39.976267994 +0000 UTC" Jul 21 13:44:03.354925 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:03.354385 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14556h51m36.62188826s" Jul 21 13:44:03.529294 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:03.528070 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:03.529294 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:03.528114 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:03.529294 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:03.528553 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:03.529294 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:03.528562 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:03.568765 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:03.568694 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" event={"ID":"6ccec470417826b6e67569d0c3ca8973","Type":"ContainerStarted","Data":"52d5f02f374d8756a5a73cf1a7b832b9fe15d60703a299af5146ea2baad7347a"} Jul 21 13:44:04.527155 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:04.526348 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:04.527155 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:04.526511 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:04.588294 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:04.588244 2567 generic.go:356] "Generic (PLEG): container finished" podID="093d700f61b33cea8abfc261e9b28005" containerID="9e1e99e050b4bbcd9f49d95ec2843321fe220688bb48c8d13ea4c8208c07ed65" exitCode=0 Jul 21 13:44:04.589685 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:04.589651 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" event={"ID":"093d700f61b33cea8abfc261e9b28005","Type":"ContainerDied","Data":"9e1e99e050b4bbcd9f49d95ec2843321fe220688bb48c8d13ea4c8208c07ed65"} Jul 21 13:44:04.608579 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:04.608512 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-212.ec2.internal" podStartSLOduration=3.6084746389999998 podStartE2EDuration="3.608474639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:03.588151407 +0000 UTC m=+4.739992522" watchObservedRunningTime="2026-07-21 13:44:04.608474639 +0000 UTC m=+5.760315755" Jul 21 13:44:05.033651 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:05.033614 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs\") pod \"network-metrics-daemon-xlpnd\" (UID: \"d08982d1-044f-462c-a84a-d4c65f1cbbef\") " pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:05.033860 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:05.033671 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhhff\" (UniqueName: \"kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff\") pod \"network-check-target-2cps2\" (UID: \"4cfe6a65-9649-46d6-a3c2-723dc4e9e218\") " pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:05.033860 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:05.033727 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:05.033976 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:05.033862 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:05.033976 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:05.033929 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret podName:093b3a4b-ec79-4a4d-a86f-05ea62a88e2c nodeName:}" failed. No retries permitted until 2026-07-21 13:44:09.033908607 +0000 UTC m=+10.185749700 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret") pod "global-pull-secret-syncer-xvrvl" (UID: "093b3a4b-ec79-4a4d-a86f-05ea62a88e2c") : object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:05.034397 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:05.034343 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:05.034397 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:05.034393 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs podName:d08982d1-044f-462c-a84a-d4c65f1cbbef nodeName:}" failed. No retries permitted until 2026-07-21 13:44:09.034377853 +0000 UTC m=+10.186218945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs") pod "network-metrics-daemon-xlpnd" (UID: "d08982d1-044f-462c-a84a-d4c65f1cbbef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:05.034554 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:05.034473 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 13:44:05.034554 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:05.034488 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 13:44:05.034554 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:05.034501 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xhhff for pod openshift-network-diagnostics/network-check-target-2cps2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:05.034554 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:05.034536 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff podName:4cfe6a65-9649-46d6-a3c2-723dc4e9e218 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:09.034525374 +0000 UTC m=+10.186366478 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhhff" (UniqueName: "kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff") pod "network-check-target-2cps2" (UID: "4cfe6a65-9649-46d6-a3c2-723dc4e9e218") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:05.528356 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:05.528323 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:05.528816 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:05.528458 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:05.530217 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:05.530158 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:05.530315 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:05.530261 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:05.599686 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:05.599646 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" event={"ID":"093d700f61b33cea8abfc261e9b28005","Type":"ContainerStarted","Data":"40ec8ad790150863cef762ee3236c41b7233211ca51da1f28ce700394873ffe9"} Jul 21 13:44:06.526669 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:06.526594 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:06.526858 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:06.526755 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:07.528925 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:07.528889 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:07.529390 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:07.529031 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:07.529390 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:07.529092 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:07.529390 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:07.529155 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:08.526596 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:08.526563 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:08.526807 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:08.526716 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:09.068078 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:09.067578 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:09.068078 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:09.067660 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs\") pod \"network-metrics-daemon-xlpnd\" (UID: \"d08982d1-044f-462c-a84a-d4c65f1cbbef\") " pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:09.068078 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:09.067710 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhhff\" (UniqueName: \"kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff\") pod \"network-check-target-2cps2\" (UID: \"4cfe6a65-9649-46d6-a3c2-723dc4e9e218\") " pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:09.068078 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:09.067836 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:09.068078 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:09.067908 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:09.068078 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:09.067930 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret podName:093b3a4b-ec79-4a4d-a86f-05ea62a88e2c nodeName:}" failed. No retries permitted until 2026-07-21 13:44:17.067904265 +0000 UTC m=+18.219745371 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret") pod "global-pull-secret-syncer-xvrvl" (UID: "093b3a4b-ec79-4a4d-a86f-05ea62a88e2c") : object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:09.068078 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:09.067961 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs podName:d08982d1-044f-462c-a84a-d4c65f1cbbef nodeName:}" failed. No retries permitted until 2026-07-21 13:44:17.067942888 +0000 UTC m=+18.219783992 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs") pod "network-metrics-daemon-xlpnd" (UID: "d08982d1-044f-462c-a84a-d4c65f1cbbef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:09.068078 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:09.067850 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 13:44:09.068078 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:09.067986 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 13:44:09.068078 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:09.068000 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xhhff for pod openshift-network-diagnostics/network-check-target-2cps2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:09.068078 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:09.068038 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff podName:4cfe6a65-9649-46d6-a3c2-723dc4e9e218 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:17.068026204 +0000 UTC m=+18.219867306 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhhff" (UniqueName: "kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff") pod "network-check-target-2cps2" (UID: "4cfe6a65-9649-46d6-a3c2-723dc4e9e218") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:09.527722 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:09.527046 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:09.527722 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:09.527167 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:09.527722 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:09.527518 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:09.527722 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:09.527616 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:10.527423 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:10.526820 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:10.527423 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:10.527003 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:11.526244 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:11.526208 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:11.526411 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:11.526208 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:11.526411 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:11.526388 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:11.526528 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:11.526451 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:12.526388 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:12.526348 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:12.526874 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:12.526496 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:13.527056 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:13.527018 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:13.527444 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:13.527138 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:13.527444 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:13.527034 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:13.527535 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:13.527488 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:14.526235 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:14.526197 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:14.526411 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:14.526333 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:15.527086 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:15.527044 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:15.527086 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:15.527081 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:15.527540 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:15.527217 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:15.527540 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:15.527350 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:16.526936 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:16.526897 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:16.527145 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:16.527026 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:17.130935 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:17.130890 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhhff\" (UniqueName: \"kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff\") pod \"network-check-target-2cps2\" (UID: \"4cfe6a65-9649-46d6-a3c2-723dc4e9e218\") " pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:17.131087 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:17.130959 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:17.131087 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:17.131016 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs\") pod \"network-metrics-daemon-xlpnd\" (UID: \"d08982d1-044f-462c-a84a-d4c65f1cbbef\") " pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:17.131087 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:17.131059 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 13:44:17.131087 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:17.131085 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 13:44:17.131263 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:17.131098 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:17.131263 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:17.131115 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:17.131263 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:17.131154 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs podName:d08982d1-044f-462c-a84a-d4c65f1cbbef nodeName:}" failed. No retries permitted until 2026-07-21 13:44:33.131139124 +0000 UTC m=+34.282980213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs") pod "network-metrics-daemon-xlpnd" (UID: "d08982d1-044f-462c-a84a-d4c65f1cbbef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:17.131263 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:17.131099 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xhhff for pod openshift-network-diagnostics/network-check-target-2cps2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:17.131263 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:17.131184 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret podName:093b3a4b-ec79-4a4d-a86f-05ea62a88e2c nodeName:}" failed. No retries permitted until 2026-07-21 13:44:33.131165138 +0000 UTC m=+34.283006232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret") pod "global-pull-secret-syncer-xvrvl" (UID: "093b3a4b-ec79-4a4d-a86f-05ea62a88e2c") : object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:17.131263 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:17.131221 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff podName:4cfe6a65-9649-46d6-a3c2-723dc4e9e218 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:33.131208109 +0000 UTC m=+34.283049210 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhhff" (UniqueName: "kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff") pod "network-check-target-2cps2" (UID: "4cfe6a65-9649-46d6-a3c2-723dc4e9e218") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:17.526798 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:17.526760 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:17.526983 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:17.526755 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:17.526983 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:17.526890 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:17.527082 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:17.526985 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:18.526602 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:18.526564 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:18.527115 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:18.526733 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:19.527067 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:19.527025 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:19.527561 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:19.527054 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:19.527561 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:19.527147 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:19.527561 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:19.527187 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:20.526570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:20.526535 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:20.526744 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:20.526665 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:21.526805 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:21.526777 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:21.527137 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:21.526786 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:21.527137 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:21.526910 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:21.527137 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:21.526983 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:22.527360 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.527030 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:22.528093 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:22.527446 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:22.643285 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.643237 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" event={"ID":"c694404b-a00c-435e-8d91-695f127abbb0","Type":"ContainerStarted","Data":"cca131579cae8e307a7297d0c1171d1e545d1ff838fc8ecba5a95b89008ebcd7"} Jul 21 13:44:22.643449 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.643342 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" event={"ID":"c694404b-a00c-435e-8d91-695f127abbb0","Type":"ContainerStarted","Data":"4a9c79bab7187f4e103e287871d6b9e7a302a5e7d1b6f7ca71f6a6c40a3cd0ce"} Jul 21 13:44:22.643449 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.643398 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" event={"ID":"c694404b-a00c-435e-8d91-695f127abbb0","Type":"ContainerStarted","Data":"def29f835c7029703d5454cbba3b5d75e40c0cc4c1245e2547ffd6d5dbde27e0"} Jul 21 13:44:22.643449 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.643445 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" event={"ID":"c694404b-a00c-435e-8d91-695f127abbb0","Type":"ContainerStarted","Data":"304e657442347e72d99e6a36ed388037e633f0e000b75ce1e3416dca33021045"} Jul 21 13:44:22.643563 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.643484 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" event={"ID":"c694404b-a00c-435e-8d91-695f127abbb0","Type":"ContainerStarted","Data":"9980ada07f2feca64f05c4d74a23c7e56219f4af3383e6d4c718cf79b009476f"} Jul 21 13:44:22.643563 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.643523 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" event={"ID":"c694404b-a00c-435e-8d91-695f127abbb0","Type":"ContainerStarted","Data":"31b4c5cb9fb6f405c9d8b149d236d1297163843aa0ed76fa19722c7d72fb432c"} Jul 21 13:44:22.644624 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.644603 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" event={"ID":"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3","Type":"ContainerStarted","Data":"4dc7e8f2501b8b6949a07f303d8f72598929915accb2c9d491c29d5befffed99"} Jul 21 13:44:22.645920 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.645900 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-js4ds" event={"ID":"2b19a38a-ffc0-4c33-914c-0eb1b2c67b71","Type":"ContainerStarted","Data":"bdf9f930fc3e7677534edf34367b58ae5c00114000b8cca8123c4a8115fb781f"} Jul 21 13:44:22.647357 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.647337 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bbnk6" event={"ID":"f26b564f-04fd-4e02-ade5-45adcf879547","Type":"ContainerStarted","Data":"0adea8b450836a7b195e4515d165751de00fed5ab75576702a4c874d4c396eab"} Jul 21 13:44:22.648791 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.648771 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wccx6" event={"ID":"554c4657-e117-4fbb-8b28-a71edec5e054","Type":"ContainerStarted","Data":"56663b90114f3c785d6d9b46dc73954d4e88a4db3fb5c349dbeda471224f9b68"} Jul 21 13:44:22.650243 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.650215 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5vvn4" event={"ID":"67843201-85e5-47e7-8100-c1bdecbefcfe","Type":"ContainerStarted","Data":"4edf8588f4844bd7227c6c570f1b64c8c54caeb0f45f07bcf58d7b1826776070"} Jul 21 13:44:22.651610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.651588 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5zc66" event={"ID":"95b64ba6-e701-4efb-bc64-74565efa56e7","Type":"ContainerStarted","Data":"5a656cf0f1f10451a4581a2eec9ddcd4d8b73dafd5e87549a65ccbdf93bacb59"} Jul 21 13:44:22.653110 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.653092 2567 generic.go:356] "Generic (PLEG): container finished" podID="c2a5ace1-fdd3-4841-992a-a0f34c621d83" containerID="a6f7bd658c894c4c677f2dde3a1b26aed7c2ea4066ce563c6edd7e396d868aa0" exitCode=0 Jul 21 13:44:22.653182 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.653118 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t26mf" event={"ID":"c2a5ace1-fdd3-4841-992a-a0f34c621d83","Type":"ContainerDied","Data":"a6f7bd658c894c4c677f2dde3a1b26aed7c2ea4066ce563c6edd7e396d868aa0"} Jul 21 13:44:22.660739 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.660679 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-js4ds" podStartSLOduration=7.424865361 podStartE2EDuration="22.660667283s" totalImagesPullingTime="15.235801922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:22.660442635 +0000 UTC m=+23.812283744" watchObservedRunningTime="2026-07-21 13:44:22.660667283 +0000 UTC m=+23.812508391" Jul 21 13:44:22.660840 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.660768 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-212.ec2.internal" podStartSLOduration=21.660763512 podStartE2EDuration="21.660763512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:05.617164162 +0000 UTC m=+6.769005271" watchObservedRunningTime="2026-07-21 13:44:22.660763512 +0000 UTC m=+23.812604620" Jul 21 13:44:22.695297 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.695236 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wccx6" podStartSLOduration=2.854522705 podStartE2EDuration="22.695220229s" totalImagesPullingTime="19.840697524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:22.695072104 +0000 UTC m=+23.846913214" watchObservedRunningTime="2026-07-21 13:44:22.695220229 +0000 UTC m=+23.847061337" Jul 21 13:44:22.710071 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.710020 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bbnk6" podStartSLOduration=3.858440695 podStartE2EDuration="23.710003s" totalImagesPullingTime="19.851562305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:22.709530283 +0000 UTC m=+23.861371395" watchObservedRunningTime="2026-07-21 13:44:22.710003 +0000 UTC m=+23.861844109" Jul 21 13:44:22.738147 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.738088 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5zc66" podStartSLOduration=2.8536131190000003 podStartE2EDuration="22.738070396s" totalImagesPullingTime="19.884457277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:22.737582345 +0000 UTC m=+23.889423453" watchObservedRunningTime="2026-07-21 13:44:22.738070396 +0000 UTC m=+23.889911505" Jul 21 13:44:22.767727 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.767659 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5vvn4" podStartSLOduration=3.070209797 podStartE2EDuration="22.767645026s" totalImagesPullingTime="19.697435229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:22.767073478 +0000 UTC m=+23.918914587" watchObservedRunningTime="2026-07-21 13:44:22.767645026 +0000 UTC m=+23.919486134" Jul 21 13:44:22.973920 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.973882 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5vvn4" Jul 21 13:44:22.974734 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:22.974688 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5vvn4" Jul 21 13:44:23.526544 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:23.526506 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:23.526761 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:23.526510 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:23.526761 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:23.526655 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:23.526889 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:23.526793 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:23.657584 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:23.657517 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9hn22" event={"ID":"07055c88-a7fe-4387-b48f-b4eed49ed54d","Type":"ContainerStarted","Data":"8d22a455c38da97d2cf0b680d98cb56edce2c3c5e4cd9bf71de8fa25993243dd"} Jul 21 13:44:23.658136 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:23.657785 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5vvn4" Jul 21 13:44:23.658718 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:23.658676 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5vvn4" Jul 21 13:44:23.675096 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:23.675028 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9hn22" podStartSLOduration=9.379482798 podStartE2EDuration="24.675008144s" totalImagesPullingTime="15.295525346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:23.674855128 +0000 UTC m=+24.826696239" watchObservedRunningTime="2026-07-21 13:44:23.675008144 +0000 UTC m=+24.826849321" Jul 21 13:44:24.526746 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:24.526684 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:24.526933 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:24.526847 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:24.766666 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:24.766620 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 13:44:25.395339 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:25.395197 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T13:44:24.766644153Z","UUID":"129e7488-ee40-46ee-b842-dd8f4403d243","Handler":null,"Name":"","Endpoint":""} Jul 21 13:44:25.397329 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:25.397307 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 13:44:25.397504 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:25.397356 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 13:44:25.531423 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:25.530988 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:25.531423 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:25.531004 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:25.531423 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:25.531116 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:25.531756 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:25.531536 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:25.675914 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:25.675838 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" event={"ID":"c694404b-a00c-435e-8d91-695f127abbb0","Type":"ContainerStarted","Data":"a2cf1ee54b68dffd0144471b86353d0cc436e338014336fdf0a3634c67046960"} Jul 21 13:44:25.679030 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:25.678991 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" event={"ID":"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3","Type":"ContainerStarted","Data":"1476f248038a8328ce77ac07c4153b8fcb9872a284c7c3182c1cc120eb36ffde"} Jul 21 13:44:26.526901 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:26.526865 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:26.527399 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:26.526993 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:27.526773 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:27.526556 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:27.526946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:27.526590 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:27.526946 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:27.526857 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:27.526946 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:27.526920 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:28.527254 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:28.527048 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:28.528169 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:28.527353 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:28.687202 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:28.687162 2567 generic.go:356] "Generic (PLEG): container finished" podID="c2a5ace1-fdd3-4841-992a-a0f34c621d83" containerID="e712e4e9f2aa38e04c0df5ff7f6920767707c3b7bc20e0c60c14a18ad3078d17" exitCode=0 Jul 21 13:44:28.687377 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:28.687248 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t26mf" event={"ID":"c2a5ace1-fdd3-4841-992a-a0f34c621d83","Type":"ContainerDied","Data":"e712e4e9f2aa38e04c0df5ff7f6920767707c3b7bc20e0c60c14a18ad3078d17"} Jul 21 13:44:28.691464 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:28.691426 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" event={"ID":"c694404b-a00c-435e-8d91-695f127abbb0","Type":"ContainerStarted","Data":"4710da69a0de8c4ba46dd2ec3d2407343e96cf6052627c5ea9669a97b4560944"} Jul 21 13:44:28.691748 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:28.691729 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:28.693794 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:28.693764 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" event={"ID":"d434a8ee-ed0c-41a7-bdc9-b804f43c6dd3","Type":"ContainerStarted","Data":"12c404c4b5a1518523c7d76b34ad262db1cf92b81502ff5757087021ad401e79"} Jul 21 13:44:28.708742 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:28.708714 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:28.806144 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:28.806009 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" podStartSLOduration=9.761123541 podStartE2EDuration="29.805987894s" totalImagesPullingTime="20.044864353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:28.804149289 +0000 UTC m=+29.955990412" watchObservedRunningTime="2026-07-21 13:44:28.805987894 +0000 UTC m=+29.957829002" Jul 21 13:44:28.832184 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:28.832137 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpnw4" podStartSLOduration=3.193610213 podStartE2EDuration="28.83212085s" totalImagesPullingTime="25.638510637s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:28.831893279 +0000 UTC m=+29.983734390" watchObservedRunningTime="2026-07-21 13:44:28.83212085 +0000 UTC m=+29.983961958" Jul 21 13:44:29.527925 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:29.527875 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:29.528371 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:29.527901 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:29.528371 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:29.528019 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:29.528371 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:29.528064 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:29.696566 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:29.696537 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:29.696773 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:29.696630 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:29.713102 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:29.713061 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:44:30.252614 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:30.252577 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xvrvl"] Jul 21 13:44:30.252799 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:30.252693 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:30.252875 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:30.252819 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:30.255963 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:30.255938 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2cps2"] Jul 21 13:44:30.256103 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:30.256031 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:30.256162 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:30.256121 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:30.260794 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:30.260769 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xlpnd"] Jul 21 13:44:30.260895 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:30.260875 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:30.260981 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:30.260962 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:30.701009 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:30.700978 2567 generic.go:356] "Generic (PLEG): container finished" podID="c2a5ace1-fdd3-4841-992a-a0f34c621d83" containerID="066d2cb6c1a97367aba6ad954e3171d0daf9d622a54f3aec0d45fbbb1906f176" exitCode=0 Jul 21 13:44:30.701401 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:30.701053 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t26mf" event={"ID":"c2a5ace1-fdd3-4841-992a-a0f34c621d83","Type":"ContainerDied","Data":"066d2cb6c1a97367aba6ad954e3171d0daf9d622a54f3aec0d45fbbb1906f176"} Jul 21 13:44:31.526577 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:31.526479 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:31.526759 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:31.526586 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:32.527232 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:32.527021 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:32.527791 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:32.527023 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:32.527791 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:32.527346 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvrvl" podUID="093b3a4b-ec79-4a4d-a86f-05ea62a88e2c" Jul 21 13:44:32.527791 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:32.527391 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlpnd" podUID="d08982d1-044f-462c-a84a-d4c65f1cbbef" Jul 21 13:44:32.708879 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:32.708836 2567 generic.go:356] "Generic (PLEG): container finished" podID="c2a5ace1-fdd3-4841-992a-a0f34c621d83" containerID="2b7f33ecd703e20009b62ed74e3b451e4e4bd83581ace9464b7030376f20014b" exitCode=0 Jul 21 13:44:32.709082 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:32.708910 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t26mf" event={"ID":"c2a5ace1-fdd3-4841-992a-a0f34c621d83","Type":"ContainerDied","Data":"2b7f33ecd703e20009b62ed74e3b451e4e4bd83581ace9464b7030376f20014b"} Jul 21 13:44:33.146182 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:33.146135 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs\") pod \"network-metrics-daemon-xlpnd\" (UID: \"d08982d1-044f-462c-a84a-d4c65f1cbbef\") " pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:33.146182 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:33.146182 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhhff\" (UniqueName: \"kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff\") pod \"network-check-target-2cps2\" (UID: \"4cfe6a65-9649-46d6-a3c2-723dc4e9e218\") " pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:33.146433 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:33.146308 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:33.146433 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:33.146322 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:33.146433 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:33.146334 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 13:44:33.146433 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:33.146367 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs podName:d08982d1-044f-462c-a84a-d4c65f1cbbef nodeName:}" failed. No retries permitted until 2026-07-21 13:45:05.146351727 +0000 UTC m=+66.298192829 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs") pod "network-metrics-daemon-xlpnd" (UID: "d08982d1-044f-462c-a84a-d4c65f1cbbef") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 13:44:33.146433 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:33.146378 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 13:44:33.146433 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:33.146394 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:33.146635 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:33.146445 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret podName:093b3a4b-ec79-4a4d-a86f-05ea62a88e2c nodeName:}" failed. No retries permitted until 2026-07-21 13:45:05.146433709 +0000 UTC m=+66.298274797 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret") pod "global-pull-secret-syncer-xvrvl" (UID: "093b3a4b-ec79-4a4d-a86f-05ea62a88e2c") : object "kube-system"/"original-pull-secret" not registered Jul 21 13:44:33.146635 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:33.146397 2567 projected.go:196] Error preparing data for projected volume kube-api-access-xhhff for pod openshift-network-diagnostics/network-check-target-2cps2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:33.146635 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:33.146482 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff podName:4cfe6a65-9649-46d6-a3c2-723dc4e9e218 nodeName:}" failed. No retries permitted until 2026-07-21 13:45:05.146472681 +0000 UTC m=+66.298313785 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhhff" (UniqueName: "kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff") pod "network-check-target-2cps2" (UID: "4cfe6a65-9649-46d6-a3c2-723dc4e9e218") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 13:44:33.526589 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:33.526552 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:33.526900 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:33.526686 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2cps2" podUID="4cfe6a65-9649-46d6-a3c2-723dc4e9e218" Jul 21 13:44:34.147255 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.147218 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-212.ec2.internal" event="NodeReady" Jul 21 13:44:34.147733 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.147421 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 13:44:34.183905 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.183864 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh"] Jul 21 13:44:34.188000 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.187945 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" Jul 21 13:44:34.190014 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.189115 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-65wbr"] Jul 21 13:44:34.193316 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.193283 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5565c7c56b-9stjh"] Jul 21 13:44:34.193472 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.193405 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.196302 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.196261 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jjh47\"" Jul 21 13:44:34.196501 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.196260 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 21 13:44:34.196574 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.196268 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 21 13:44:34.196631 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.196457 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 21 13:44:34.196807 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.196790 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 21 13:44:34.197075 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.197053 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 21 13:44:34.197648 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.197628 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f2v6c"] Jul 21 13:44:34.198117 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.197786 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.198571 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.198438 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 13:44:34.198973 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.198955 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 21 13:44:34.199426 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.199411 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 13:44:34.199652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.199626 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xhbc4\"" Jul 21 13:44:34.200288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.200264 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 13:44:34.200543 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.200514 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 21 13:44:34.202426 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.202231 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 13:44:34.202426 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.202231 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 21 13:44:34.202426 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.202277 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 21 13:44:34.202426 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.202320 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 21 13:44:34.203111 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.203093 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-cf9ph\"" Jul 21 13:44:34.203686 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.203410 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq"] Jul 21 13:44:34.204416 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.204320 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f2v6c" Jul 21 13:44:34.207984 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.207330 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 21 13:44:34.207984 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.207651 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9"] Jul 21 13:44:34.207984 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.207944 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:34.209880 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.209558 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 21 13:44:34.209880 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.209826 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 21 13:44:34.210079 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.210061 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gl87q\"" Jul 21 13:44:34.212851 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.212136 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-789fb75d98-pqr2q"] Jul 21 13:44:34.212851 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.212200 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 13:44:34.212851 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.212238 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 21 13:44:34.212851 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.212249 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 21 13:44:34.212851 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.212495 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gz5rr\"" Jul 21 13:44:34.212851 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.212506 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 13:44:34.212851 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.212516 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" Jul 21 13:44:34.216008 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.215982 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8"] Jul 21 13:44:34.216184 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.216160 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 21 13:44:34.216291 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.216241 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.217306 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.217283 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 21 13:44:34.218397 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.217582 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 21 13:44:34.218397 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.218032 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 21 13:44:34.221785 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.219068 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-l4f44\"" Jul 21 13:44:34.221785 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.220246 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 21 13:44:34.221785 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.220843 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mm57m\"" Jul 21 13:44:34.221785 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.221734 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 21 13:44:34.222324 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.222305 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 21 13:44:34.229582 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.229556 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-pdjqz"] Jul 21 13:44:34.230035 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.230013 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:44:34.233244 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.233220 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 21 13:44:34.233618 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.233596 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 21 13:44:34.234089 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.234063 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 21 13:44:34.234293 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.234179 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kfz4h\"" Jul 21 13:44:34.235762 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.235738 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 21 13:44:34.236831 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.236809 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh"] Jul 21 13:44:34.236930 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.236853 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l"] Jul 21 13:44:34.236986 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.236931 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.240588 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.240406 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-glfqj"] Jul 21 13:44:34.241444 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.240850 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:44:34.246602 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.242635 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 21 13:44:34.246602 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.242601 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 21 13:44:34.246602 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.242763 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-zh6z9\"" Jul 21 13:44:34.246602 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.242908 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 21 13:44:34.246602 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.243087 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 21 13:44:34.246602 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.244245 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 21 13:44:34.246602 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.245239 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dtmfw"] Jul 21 13:44:34.246602 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.246401 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-glfqj" Jul 21 13:44:34.247675 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.247651 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 21 13:44:34.249995 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.249969 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-2ck97\"" Jul 21 13:44:34.250434 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.250412 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 13:44:34.250543 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.250508 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 13:44:34.250811 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.250794 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qjfhr\"" Jul 21 13:44:34.251409 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.251384 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 21 13:44:34.252807 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.252790 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:34.253332 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.253316 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p8hzg"] Jul 21 13:44:34.257098 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.256797 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-pdjqz"] Jul 21 13:44:34.257098 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.256838 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-65wbr"] Jul 21 13:44:34.257098 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.256858 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f2v6c"] Jul 21 13:44:34.257098 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.256878 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8"] Jul 21 13:44:34.257098 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.256904 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9"] Jul 21 13:44:34.257098 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.256923 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5565c7c56b-9stjh"] Jul 21 13:44:34.257098 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.256940 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq"] Jul 21 13:44:34.257098 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.256957 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l"] Jul 21 13:44:34.257098 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.256973 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-glfqj"] Jul 21 13:44:34.257098 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.256999 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-789fb75d98-pqr2q"] Jul 21 13:44:34.257098 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.257015 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dtmfw"] Jul 21 13:44:34.257098 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.257040 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p8hzg"] Jul 21 13:44:34.257819 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.257149 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p8hzg" Jul 21 13:44:34.257819 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.257309 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 13:44:34.257819 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.257350 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 13:44:34.257819 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.257309 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2pd27\"" Jul 21 13:44:34.259996 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.259956 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 13:44:34.260189 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.260025 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dghlb\"" Jul 21 13:44:34.260302 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.260233 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 13:44:34.260302 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.260233 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 13:44:34.354550 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.354516 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8d80093e-2150-45d5-b78a-a96386daf7e9-snapshots\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.354550 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.354555 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57qxh\" (UniqueName: \"kubernetes.io/projected/80d7f298-9397-475b-951d-baf26856c339-kube-api-access-57qxh\") pod \"kube-storage-version-migrator-operator-7658d9b749-p56gh\" (UID: \"80d7f298-9397-475b-951d-baf26856c339\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" Jul 21 13:44:34.354798 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.354576 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3b4040af-d0ae-46c2-ab55-880e8f31af17-ca-trust-extracted\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.354798 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.354598 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd83129f-d2e9-4efc-99be-4bc5af88e534-trusted-ca\") pod \"console-operator-565b7b569d-pdjqz\" (UID: \"cd83129f-d2e9-4efc-99be-4bc5af88e534\") " pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.354798 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.354634 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:34.354798 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.354692 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-certificates\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.354798 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.354757 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.354798 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.354785 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/108a7342-129a-4511-bc08-2b15d4407afa-config\") pod \"service-ca-operator-5bcbc94c78-zpdl9\" (UID: \"108a7342-129a-4511-bc08-2b15d4407afa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" Jul 21 13:44:34.355054 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.354816 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99bnb\" (UniqueName: \"kubernetes.io/projected/cd83129f-d2e9-4efc-99be-4bc5af88e534-kube-api-access-99bnb\") pod \"console-operator-565b7b569d-pdjqz\" (UID: \"cd83129f-d2e9-4efc-99be-4bc5af88e534\") " pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.355054 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.354843 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kfq8l\" (UID: \"6b173759-de6d-43c8-adaa-41a0df593ed2\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:44:34.355054 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.354869 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d80093e-2150-45d5-b78a-a96386daf7e9-service-ca-bundle\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.355054 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.354912 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vwll\" (UniqueName: \"kubernetes.io/projected/53a477e4-4535-4679-acbb-197a47d3e114-kube-api-access-8vwll\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:34.355054 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.354992 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvhsg\" (UniqueName: \"kubernetes.io/projected/9ea20305-07d4-404e-91f7-fe2c5ef68598-kube-api-access-nvhsg\") pod \"cluster-samples-operator-79b7c55f87-m57g8\" (UID: \"9ea20305-07d4-404e-91f7-fe2c5ef68598\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:44:34.355054 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355027 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jchn\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-kube-api-access-2jchn\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.355054 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355053 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-bound-sa-token\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.355360 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355072 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53a477e4-4535-4679-acbb-197a47d3e114-tmp-dir\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:34.355360 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355087 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grcqg\" (UniqueName: \"kubernetes.io/projected/fe984719-83dd-497e-8f59-9fd42fabbde3-kube-api-access-grcqg\") pod \"network-check-source-7bc5b6bf85-glfqj\" (UID: \"fe984719-83dd-497e-8f59-9fd42fabbde3\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-glfqj" Jul 21 13:44:34.355360 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355121 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cd83129f-d2e9-4efc-99be-4bc5af88e534-serving-cert\") pod \"console-operator-565b7b569d-pdjqz\" (UID: \"cd83129f-d2e9-4efc-99be-4bc5af88e534\") " pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.355360 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355153 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/80d7f298-9397-475b-951d-baf26856c339-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-p56gh\" (UID: \"80d7f298-9397-475b-951d-baf26856c339\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" Jul 21 13:44:34.355360 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355215 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jnlw\" (UniqueName: \"kubernetes.io/projected/8d80093e-2150-45d5-b78a-a96386daf7e9-kube-api-access-2jnlw\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.355360 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355325 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:34.355604 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355361 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d80093e-2150-45d5-b78a-a96386daf7e9-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.355604 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355400 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhz76\" (UniqueName: \"kubernetes.io/projected/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-kube-api-access-qhz76\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:34.355604 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355436 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3b4040af-d0ae-46c2-ab55-880e8f31af17-installation-pull-secrets\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.355604 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355493 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-stats-auth\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.355604 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355530 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d80093e-2150-45d5-b78a-a96386daf7e9-serving-cert\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.355604 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355557 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3b4040af-d0ae-46c2-ab55-880e8f31af17-image-registry-private-configuration\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.355604 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355591 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8rw7\" (UniqueName: \"kubernetes.io/projected/a4ff8e96-9fe5-4f87-8f9c-a2d740589863-kube-api-access-q8rw7\") pod \"ingress-canary-p8hzg\" (UID: \"a4ff8e96-9fe5-4f87-8f9c-a2d740589863\") " pod="openshift-ingress-canary/ingress-canary-p8hzg" Jul 21 13:44:34.355903 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355662 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.355903 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355719 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd83129f-d2e9-4efc-99be-4bc5af88e534-config\") pod \"console-operator-565b7b569d-pdjqz\" (UID: \"cd83129f-d2e9-4efc-99be-4bc5af88e534\") " pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.355903 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355776 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/108a7342-129a-4511-bc08-2b15d4407afa-serving-cert\") pod \"service-ca-operator-5bcbc94c78-zpdl9\" (UID: \"108a7342-129a-4511-bc08-2b15d4407afa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" Jul 21 13:44:34.355903 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355804 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-m57g8\" (UID: \"9ea20305-07d4-404e-91f7-fe2c5ef68598\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:44:34.355903 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355829 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm9jj\" (UniqueName: \"kubernetes.io/projected/c7b2425c-081b-45a4-86e2-a0445311d69c-kube-api-access-vm9jj\") pod \"volume-data-source-validator-78cf4c6f6-f2v6c\" (UID: \"c7b2425c-081b-45a4-86e2-a0445311d69c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f2v6c" Jul 21 13:44:34.355903 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355850 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.355903 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355869 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a4ff8e96-9fe5-4f87-8f9c-a2d740589863-cert\") pod \"ingress-canary-p8hzg\" (UID: \"a4ff8e96-9fe5-4f87-8f9c-a2d740589863\") " pod="openshift-ingress-canary/ingress-canary-p8hzg" Jul 21 13:44:34.355903 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355897 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-default-certificate\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.356518 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355920 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d80093e-2150-45d5-b78a-a96386daf7e9-tmp\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.356518 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355958 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvvxm\" (UniqueName: \"kubernetes.io/projected/108a7342-129a-4511-bc08-2b15d4407afa-kube-api-access-qvvxm\") pod \"service-ca-operator-5bcbc94c78-zpdl9\" (UID: \"108a7342-129a-4511-bc08-2b15d4407afa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" Jul 21 13:44:34.356518 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.355982 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3b4040af-d0ae-46c2-ab55-880e8f31af17-trusted-ca\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.356518 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.356008 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6b173759-de6d-43c8-adaa-41a0df593ed2-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-kfq8l\" (UID: \"6b173759-de6d-43c8-adaa-41a0df593ed2\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:44:34.356518 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.356042 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/80d7f298-9397-475b-951d-baf26856c339-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-p56gh\" (UID: \"80d7f298-9397-475b-951d-baf26856c339\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" Jul 21 13:44:34.356518 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.356069 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67frg\" (UniqueName: \"kubernetes.io/projected/d29c3f6d-9285-48fa-979b-b71b91753fc1-kube-api-access-67frg\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.356518 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.356093 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:34.356518 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.356116 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/53a477e4-4535-4679-acbb-197a47d3e114-config-volume\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:34.395071 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.395031 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:34.456800 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.456765 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/53a477e4-4535-4679-acbb-197a47d3e114-config-volume\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:34.456800 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.456808 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8d80093e-2150-45d5-b78a-a96386daf7e9-snapshots\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.457035 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.456833 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57qxh\" (UniqueName: \"kubernetes.io/projected/80d7f298-9397-475b-951d-baf26856c339-kube-api-access-57qxh\") pod \"kube-storage-version-migrator-operator-7658d9b749-p56gh\" (UID: \"80d7f298-9397-475b-951d-baf26856c339\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" Jul 21 13:44:34.457035 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.456860 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3b4040af-d0ae-46c2-ab55-880e8f31af17-ca-trust-extracted\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.457035 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.456886 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd83129f-d2e9-4efc-99be-4bc5af88e534-trusted-ca\") pod \"console-operator-565b7b569d-pdjqz\" (UID: \"cd83129f-d2e9-4efc-99be-4bc5af88e534\") " pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.457035 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.456915 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:34.457035 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.456943 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-certificates\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.457035 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.456972 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.457035 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.456997 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/108a7342-129a-4511-bc08-2b15d4407afa-config\") pod \"service-ca-operator-5bcbc94c78-zpdl9\" (UID: \"108a7342-129a-4511-bc08-2b15d4407afa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" Jul 21 13:44:34.457035 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457024 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99bnb\" (UniqueName: \"kubernetes.io/projected/cd83129f-d2e9-4efc-99be-4bc5af88e534-kube-api-access-99bnb\") pod \"console-operator-565b7b569d-pdjqz\" (UID: \"cd83129f-d2e9-4efc-99be-4bc5af88e534\") " pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.457345 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457050 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kfq8l\" (UID: \"6b173759-de6d-43c8-adaa-41a0df593ed2\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:44:34.457345 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457073 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d80093e-2150-45d5-b78a-a96386daf7e9-service-ca-bundle\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.457345 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457097 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vwll\" (UniqueName: \"kubernetes.io/projected/53a477e4-4535-4679-acbb-197a47d3e114-kube-api-access-8vwll\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:34.457345 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457125 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvhsg\" (UniqueName: \"kubernetes.io/projected/9ea20305-07d4-404e-91f7-fe2c5ef68598-kube-api-access-nvhsg\") pod \"cluster-samples-operator-79b7c55f87-m57g8\" (UID: \"9ea20305-07d4-404e-91f7-fe2c5ef68598\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:44:34.457345 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457150 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jchn\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-kube-api-access-2jchn\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.457345 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457179 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-bound-sa-token\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.457345 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457204 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53a477e4-4535-4679-acbb-197a47d3e114-tmp-dir\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:34.457345 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457231 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grcqg\" (UniqueName: \"kubernetes.io/projected/fe984719-83dd-497e-8f59-9fd42fabbde3-kube-api-access-grcqg\") pod \"network-check-source-7bc5b6bf85-glfqj\" (UID: \"fe984719-83dd-497e-8f59-9fd42fabbde3\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-glfqj" Jul 21 13:44:34.457345 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457274 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cd83129f-d2e9-4efc-99be-4bc5af88e534-serving-cert\") pod \"console-operator-565b7b569d-pdjqz\" (UID: \"cd83129f-d2e9-4efc-99be-4bc5af88e534\") " pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.457345 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457291 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3b4040af-d0ae-46c2-ab55-880e8f31af17-ca-trust-extracted\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.457345 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457303 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/80d7f298-9397-475b-951d-baf26856c339-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-p56gh\" (UID: \"80d7f298-9397-475b-951d-baf26856c339\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" Jul 21 13:44:34.457784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457356 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jnlw\" (UniqueName: \"kubernetes.io/projected/8d80093e-2150-45d5-b78a-a96386daf7e9-kube-api-access-2jnlw\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.457784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457406 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:34.457784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457444 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d80093e-2150-45d5-b78a-a96386daf7e9-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.457784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457469 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhz76\" (UniqueName: \"kubernetes.io/projected/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-kube-api-access-qhz76\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:34.457784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457498 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3b4040af-d0ae-46c2-ab55-880e8f31af17-installation-pull-secrets\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.457784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457529 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-stats-auth\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.457784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457566 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8d80093e-2150-45d5-b78a-a96386daf7e9-snapshots\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.457784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457581 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d80093e-2150-45d5-b78a-a96386daf7e9-serving-cert\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.457784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457628 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3b4040af-d0ae-46c2-ab55-880e8f31af17-image-registry-private-configuration\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.457784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457660 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8rw7\" (UniqueName: \"kubernetes.io/projected/a4ff8e96-9fe5-4f87-8f9c-a2d740589863-kube-api-access-q8rw7\") pod \"ingress-canary-p8hzg\" (UID: \"a4ff8e96-9fe5-4f87-8f9c-a2d740589863\") " pod="openshift-ingress-canary/ingress-canary-p8hzg" Jul 21 13:44:34.457784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457689 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/53a477e4-4535-4679-acbb-197a47d3e114-config-volume\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:34.457784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457721 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.457784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457775 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd83129f-d2e9-4efc-99be-4bc5af88e534-config\") pod \"console-operator-565b7b569d-pdjqz\" (UID: \"cd83129f-d2e9-4efc-99be-4bc5af88e534\") " pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.458288 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.457827 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle podName:d29c3f6d-9285-48fa-979b-b71b91753fc1 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:34.95780931 +0000 UTC m=+36.109650418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle") pod "router-default-5565c7c56b-9stjh" (UID: "d29c3f6d-9285-48fa-979b-b71b91753fc1") : configmap references non-existent config key: service-ca.crt Jul 21 13:44:34.458288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457851 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/108a7342-129a-4511-bc08-2b15d4407afa-serving-cert\") pod \"service-ca-operator-5bcbc94c78-zpdl9\" (UID: \"108a7342-129a-4511-bc08-2b15d4407afa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" Jul 21 13:44:34.458288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457881 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-m57g8\" (UID: \"9ea20305-07d4-404e-91f7-fe2c5ef68598\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:44:34.458288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457911 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm9jj\" (UniqueName: \"kubernetes.io/projected/c7b2425c-081b-45a4-86e2-a0445311d69c-kube-api-access-vm9jj\") pod \"volume-data-source-validator-78cf4c6f6-f2v6c\" (UID: \"c7b2425c-081b-45a4-86e2-a0445311d69c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f2v6c" Jul 21 13:44:34.458288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457943 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.458288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457969 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a4ff8e96-9fe5-4f87-8f9c-a2d740589863-cert\") pod \"ingress-canary-p8hzg\" (UID: \"a4ff8e96-9fe5-4f87-8f9c-a2d740589863\") " pod="openshift-ingress-canary/ingress-canary-p8hzg" Jul 21 13:44:34.458288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.458000 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-default-certificate\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.458288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.458033 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d80093e-2150-45d5-b78a-a96386daf7e9-tmp\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.458288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.458079 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvvxm\" (UniqueName: \"kubernetes.io/projected/108a7342-129a-4511-bc08-2b15d4407afa-kube-api-access-qvvxm\") pod \"service-ca-operator-5bcbc94c78-zpdl9\" (UID: \"108a7342-129a-4511-bc08-2b15d4407afa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" Jul 21 13:44:34.458288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.458106 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3b4040af-d0ae-46c2-ab55-880e8f31af17-trusted-ca\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.458288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.458134 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6b173759-de6d-43c8-adaa-41a0df593ed2-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-kfq8l\" (UID: \"6b173759-de6d-43c8-adaa-41a0df593ed2\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:44:34.458288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.458175 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/80d7f298-9397-475b-951d-baf26856c339-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-p56gh\" (UID: \"80d7f298-9397-475b-951d-baf26856c339\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" Jul 21 13:44:34.458806 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.458211 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67frg\" (UniqueName: \"kubernetes.io/projected/d29c3f6d-9285-48fa-979b-b71b91753fc1-kube-api-access-67frg\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.458806 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.458242 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:34.458806 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.458288 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd83129f-d2e9-4efc-99be-4bc5af88e534-config\") pod \"console-operator-565b7b569d-pdjqz\" (UID: \"cd83129f-d2e9-4efc-99be-4bc5af88e534\") " pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.458806 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.458472 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 13:44:34.458806 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.458486 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-789fb75d98-pqr2q: secret "image-registry-tls" not found Jul 21 13:44:34.458806 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.458540 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls podName:3b4040af-d0ae-46c2-ab55-880e8f31af17 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:34.958525875 +0000 UTC m=+36.110366976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls") pod "image-registry-789fb75d98-pqr2q" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17") : secret "image-registry-tls" not found Jul 21 13:44:34.458806 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.458571 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d80093e-2150-45d5-b78a-a96386daf7e9-tmp\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.458806 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.458598 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 13:44:34.458806 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.458629 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls podName:9ea20305-07d4-404e-91f7-fe2c5ef68598 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:34.95861964 +0000 UTC m=+36.110460740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-m57g8" (UID: "9ea20305-07d4-404e-91f7-fe2c5ef68598") : secret "samples-operator-tls" not found Jul 21 13:44:34.458806 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.458788 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d80093e-2150-45d5-b78a-a96386daf7e9-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.460266 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.459751 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3b4040af-d0ae-46c2-ab55-880e8f31af17-trusted-ca\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.460266 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.459851 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 13:44:34.460266 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.459934 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls podName:53a477e4-4535-4679-acbb-197a47d3e114 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:34.959915923 +0000 UTC m=+36.111757078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls") pod "dns-default-dtmfw" (UID: "53a477e4-4535-4679-acbb-197a47d3e114") : secret "dns-default-metrics-tls" not found Jul 21 13:44:34.462412 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.460521 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-certificates\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.462412 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.460591 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:34.462412 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.460612 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 13:44:34.462412 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.460660 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs podName:d29c3f6d-9285-48fa-979b-b71b91753fc1 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:34.960646046 +0000 UTC m=+36.112487133 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs") pod "router-default-5565c7c56b-9stjh" (UID: "d29c3f6d-9285-48fa-979b-b71b91753fc1") : secret "router-metrics-certs-default" not found Jul 21 13:44:34.462412 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.460690 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 13:44:34.462412 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.460774 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls podName:37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a nodeName:}" failed. No retries permitted until 2026-07-21 13:44:34.960758513 +0000 UTC m=+36.112599604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ng6jq" (UID: "37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a") : secret "cluster-monitoring-operator-tls" not found Jul 21 13:44:34.462412 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.461167 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/108a7342-129a-4511-bc08-2b15d4407afa-config\") pod \"service-ca-operator-5bcbc94c78-zpdl9\" (UID: \"108a7342-129a-4511-bc08-2b15d4407afa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" Jul 21 13:44:34.462412 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.461182 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 13:44:34.462412 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.461234 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert podName:6b173759-de6d-43c8-adaa-41a0df593ed2 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:34.96121942 +0000 UTC m=+36.113060507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-kfq8l" (UID: "6b173759-de6d-43c8-adaa-41a0df593ed2") : secret "networking-console-plugin-cert" not found Jul 21 13:44:34.462412 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.461636 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d80093e-2150-45d5-b78a-a96386daf7e9-service-ca-bundle\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.462412 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.461930 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53a477e4-4535-4679-acbb-197a47d3e114-tmp-dir\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:34.462412 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.457965 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd83129f-d2e9-4efc-99be-4bc5af88e534-trusted-ca\") pod \"console-operator-565b7b569d-pdjqz\" (UID: \"cd83129f-d2e9-4efc-99be-4bc5af88e534\") " pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.462412 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.462029 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/80d7f298-9397-475b-951d-baf26856c339-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-p56gh\" (UID: \"80d7f298-9397-475b-951d-baf26856c339\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" Jul 21 13:44:34.463141 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.462630 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6b173759-de6d-43c8-adaa-41a0df593ed2-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-kfq8l\" (UID: \"6b173759-de6d-43c8-adaa-41a0df593ed2\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:44:34.463141 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.462693 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/108a7342-129a-4511-bc08-2b15d4407afa-serving-cert\") pod \"service-ca-operator-5bcbc94c78-zpdl9\" (UID: \"108a7342-129a-4511-bc08-2b15d4407afa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" Jul 21 13:44:34.464471 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.464441 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d80093e-2150-45d5-b78a-a96386daf7e9-serving-cert\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.464673 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.464655 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cd83129f-d2e9-4efc-99be-4bc5af88e534-serving-cert\") pod \"console-operator-565b7b569d-pdjqz\" (UID: \"cd83129f-d2e9-4efc-99be-4bc5af88e534\") " pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.465222 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.465198 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/80d7f298-9397-475b-951d-baf26856c339-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-p56gh\" (UID: \"80d7f298-9397-475b-951d-baf26856c339\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" Jul 21 13:44:34.465313 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.465243 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a4ff8e96-9fe5-4f87-8f9c-a2d740589863-cert\") pod \"ingress-canary-p8hzg\" (UID: \"a4ff8e96-9fe5-4f87-8f9c-a2d740589863\") " pod="openshift-ingress-canary/ingress-canary-p8hzg" Jul 21 13:44:34.465313 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.465251 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3b4040af-d0ae-46c2-ab55-880e8f31af17-image-registry-private-configuration\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.465422 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.465349 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-stats-auth\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.465652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.465612 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3b4040af-d0ae-46c2-ab55-880e8f31af17-installation-pull-secrets\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.465999 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.465977 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-default-certificate\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.468572 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.468549 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57qxh\" (UniqueName: \"kubernetes.io/projected/80d7f298-9397-475b-951d-baf26856c339-kube-api-access-57qxh\") pod \"kube-storage-version-migrator-operator-7658d9b749-p56gh\" (UID: \"80d7f298-9397-475b-951d-baf26856c339\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" Jul 21 13:44:34.472627 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.472582 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm9jj\" (UniqueName: \"kubernetes.io/projected/c7b2425c-081b-45a4-86e2-a0445311d69c-kube-api-access-vm9jj\") pod \"volume-data-source-validator-78cf4c6f6-f2v6c\" (UID: \"c7b2425c-081b-45a4-86e2-a0445311d69c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f2v6c" Jul 21 13:44:34.472627 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.472606 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvvxm\" (UniqueName: \"kubernetes.io/projected/108a7342-129a-4511-bc08-2b15d4407afa-kube-api-access-qvvxm\") pod \"service-ca-operator-5bcbc94c78-zpdl9\" (UID: \"108a7342-129a-4511-bc08-2b15d4407afa\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" Jul 21 13:44:34.472988 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.472971 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhz76\" (UniqueName: \"kubernetes.io/projected/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-kube-api-access-qhz76\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:34.473409 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.473388 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jnlw\" (UniqueName: \"kubernetes.io/projected/8d80093e-2150-45d5-b78a-a96386daf7e9-kube-api-access-2jnlw\") pod \"insights-operator-555d9d7885-65wbr\" (UID: \"8d80093e-2150-45d5-b78a-a96386daf7e9\") " pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.473732 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.473692 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8rw7\" (UniqueName: \"kubernetes.io/projected/a4ff8e96-9fe5-4f87-8f9c-a2d740589863-kube-api-access-q8rw7\") pod \"ingress-canary-p8hzg\" (UID: \"a4ff8e96-9fe5-4f87-8f9c-a2d740589863\") " pod="openshift-ingress-canary/ingress-canary-p8hzg" Jul 21 13:44:34.477446 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.477416 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grcqg\" (UniqueName: \"kubernetes.io/projected/fe984719-83dd-497e-8f59-9fd42fabbde3-kube-api-access-grcqg\") pod \"network-check-source-7bc5b6bf85-glfqj\" (UID: \"fe984719-83dd-497e-8f59-9fd42fabbde3\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-glfqj" Jul 21 13:44:34.477757 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.477692 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jchn\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-kube-api-access-2jchn\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.477843 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.477764 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99bnb\" (UniqueName: \"kubernetes.io/projected/cd83129f-d2e9-4efc-99be-4bc5af88e534-kube-api-access-99bnb\") pod \"console-operator-565b7b569d-pdjqz\" (UID: \"cd83129f-d2e9-4efc-99be-4bc5af88e534\") " pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.478285 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.478250 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-bound-sa-token\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.479013 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.478987 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvhsg\" (UniqueName: \"kubernetes.io/projected/9ea20305-07d4-404e-91f7-fe2c5ef68598-kube-api-access-nvhsg\") pod \"cluster-samples-operator-79b7c55f87-m57g8\" (UID: \"9ea20305-07d4-404e-91f7-fe2c5ef68598\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:44:34.481788 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.481769 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vwll\" (UniqueName: \"kubernetes.io/projected/53a477e4-4535-4679-acbb-197a47d3e114-kube-api-access-8vwll\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:34.482104 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.482082 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67frg\" (UniqueName: \"kubernetes.io/projected/d29c3f6d-9285-48fa-979b-b71b91753fc1-kube-api-access-67frg\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.502969 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.502930 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" Jul 21 13:44:34.513109 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.513050 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-65wbr" Jul 21 13:44:34.526822 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.526791 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:44:34.526822 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.526813 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:44:34.530329 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.530106 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 13:44:34.533555 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.533515 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 13:44:34.533769 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.533519 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pt8sb\"" Jul 21 13:44:34.534630 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.534609 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f2v6c" Jul 21 13:44:34.557438 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.557395 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" Jul 21 13:44:34.587537 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.587226 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:34.601948 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.599936 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-glfqj" Jul 21 13:44:34.614027 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.613498 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p8hzg" Jul 21 13:44:34.731633 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.731198 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-65wbr"] Jul 21 13:44:34.738622 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.738591 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh"] Jul 21 13:44:34.751901 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:34.745792 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d80093e_2150_45d5_b78a_a96386daf7e9.slice/crio-9909b774e96a0ddb8f6866a4eab8545c77fbda8c45475193b7bdf86f4b10f9e4 WatchSource:0}: Error finding container 9909b774e96a0ddb8f6866a4eab8545c77fbda8c45475193b7bdf86f4b10f9e4: Status 404 returned error can't find the container with id 9909b774e96a0ddb8f6866a4eab8545c77fbda8c45475193b7bdf86f4b10f9e4 Jul 21 13:44:34.757776 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.757731 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f2v6c"] Jul 21 13:44:34.795669 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.795594 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9"] Jul 21 13:44:34.799451 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:34.799420 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod108a7342_129a_4511_bc08_2b15d4407afa.slice/crio-ce0d3a9870848b11531c1e6ab8bd2266ab7e2f2781d3fba688620809ff107688 WatchSource:0}: Error finding container ce0d3a9870848b11531c1e6ab8bd2266ab7e2f2781d3fba688620809ff107688: Status 404 returned error can't find the container with id ce0d3a9870848b11531c1e6ab8bd2266ab7e2f2781d3fba688620809ff107688 Jul 21 13:44:34.842309 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.842239 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-pdjqz"] Jul 21 13:44:34.845344 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:34.845307 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd83129f_d2e9_4efc_99be_4bc5af88e534.slice/crio-6f093e2cbb97dd4cd5f955c682c69d5316f3f0968ae889ae871cc17f175bcbb7 WatchSource:0}: Error finding container 6f093e2cbb97dd4cd5f955c682c69d5316f3f0968ae889ae871cc17f175bcbb7: Status 404 returned error can't find the container with id 6f093e2cbb97dd4cd5f955c682c69d5316f3f0968ae889ae871cc17f175bcbb7 Jul 21 13:44:34.873196 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.873159 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-glfqj"] Jul 21 13:44:34.874367 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.874299 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p8hzg"] Jul 21 13:44:34.876180 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:34.876141 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe984719_83dd_497e_8f59_9fd42fabbde3.slice/crio-7e8a546e9cb7df4f38e42e7b2cc16445398fe7907ac46e2d8b26980281aa3b1d WatchSource:0}: Error finding container 7e8a546e9cb7df4f38e42e7b2cc16445398fe7907ac46e2d8b26980281aa3b1d: Status 404 returned error can't find the container with id 7e8a546e9cb7df4f38e42e7b2cc16445398fe7907ac46e2d8b26980281aa3b1d Jul 21 13:44:34.877000 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:34.876972 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ff8e96_9fe5_4f87_8f9c_a2d740589863.slice/crio-4a992589d06d1d65f4bc0b5bab38106e71d291d2c029c92b02a35d9d8bbca657 WatchSource:0}: Error finding container 4a992589d06d1d65f4bc0b5bab38106e71d291d2c029c92b02a35d9d8bbca657: Status 404 returned error can't find the container with id 4a992589d06d1d65f4bc0b5bab38106e71d291d2c029c92b02a35d9d8bbca657 Jul 21 13:44:34.964295 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.964257 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:34.964516 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.964321 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.964516 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.964353 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-m57g8\" (UID: \"9ea20305-07d4-404e-91f7-fe2c5ef68598\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:44:34.964516 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.964375 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:34.964516 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.964437 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 13:44:34.964516 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.964435 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:34.964516 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.964504 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:34.964516 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.964519 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls podName:37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a nodeName:}" failed. No retries permitted until 2026-07-21 13:44:35.964499266 +0000 UTC m=+37.116340353 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ng6jq" (UID: "37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a") : secret "cluster-monitoring-operator-tls" not found Jul 21 13:44:34.964868 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.964536 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 13:44:34.964868 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.964553 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 13:44:34.964868 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.964565 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 13:44:34.964868 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.964571 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-789fb75d98-pqr2q: secret "image-registry-tls" not found Jul 21 13:44:34.964868 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.964575 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 13:44:34.964868 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.964603 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle podName:d29c3f6d-9285-48fa-979b-b71b91753fc1 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:35.964584772 +0000 UTC m=+37.116425860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle") pod "router-default-5565c7c56b-9stjh" (UID: "d29c3f6d-9285-48fa-979b-b71b91753fc1") : configmap references non-existent config key: service-ca.crt Jul 21 13:44:34.964868 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.964627 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls podName:9ea20305-07d4-404e-91f7-fe2c5ef68598 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:35.964616274 +0000 UTC m=+37.116457367 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-m57g8" (UID: "9ea20305-07d4-404e-91f7-fe2c5ef68598") : secret "samples-operator-tls" not found Jul 21 13:44:34.964868 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.964672 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls podName:53a477e4-4535-4679-acbb-197a47d3e114 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:35.964635648 +0000 UTC m=+37.116476740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls") pod "dns-default-dtmfw" (UID: "53a477e4-4535-4679-acbb-197a47d3e114") : secret "dns-default-metrics-tls" not found Jul 21 13:44:34.964868 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.964719 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls podName:3b4040af-d0ae-46c2-ab55-880e8f31af17 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:35.964687465 +0000 UTC m=+37.116528561 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls") pod "image-registry-789fb75d98-pqr2q" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17") : secret "image-registry-tls" not found Jul 21 13:44:34.964868 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.964744 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs podName:d29c3f6d-9285-48fa-979b-b71b91753fc1 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:35.964735612 +0000 UTC m=+37.116576729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs") pod "router-default-5565c7c56b-9stjh" (UID: "d29c3f6d-9285-48fa-979b-b71b91753fc1") : secret "router-metrics-certs-default" not found Jul 21 13:44:34.964868 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:34.964840 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kfq8l\" (UID: \"6b173759-de6d-43c8-adaa-41a0df593ed2\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:44:34.965286 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.965017 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 13:44:34.965286 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:34.965067 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert podName:6b173759-de6d-43c8-adaa-41a0df593ed2 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:35.965054112 +0000 UTC m=+37.116895200 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-kfq8l" (UID: "6b173759-de6d-43c8-adaa-41a0df593ed2") : secret "networking-console-plugin-cert" not found Jul 21 13:44:35.462460 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.462334 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:35.529842 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.529236 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:44:35.533188 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.532976 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-29gkd\"" Jul 21 13:44:35.727938 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.727847 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:35.732432 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.732242 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" event={"ID":"108a7342-129a-4511-bc08-2b15d4407afa","Type":"ContainerStarted","Data":"ce0d3a9870848b11531c1e6ab8bd2266ab7e2f2781d3fba688620809ff107688"} Jul 21 13:44:35.737557 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.737447 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f2v6c" event={"ID":"c7b2425c-081b-45a4-86e2-a0445311d69c","Type":"ContainerStarted","Data":"11d27e862df915a54a152065c1bc2dbc799af04a2a9f51f051f396ff450563c8"} Jul 21 13:44:35.740396 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.740360 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" event={"ID":"cd83129f-d2e9-4efc-99be-4bc5af88e534","Type":"ContainerStarted","Data":"6f093e2cbb97dd4cd5f955c682c69d5316f3f0968ae889ae871cc17f175bcbb7"} Jul 21 13:44:35.742018 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.741943 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p8hzg" event={"ID":"a4ff8e96-9fe5-4f87-8f9c-a2d740589863","Type":"ContainerStarted","Data":"4a992589d06d1d65f4bc0b5bab38106e71d291d2c029c92b02a35d9d8bbca657"} Jul 21 13:44:35.746073 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.745990 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-65wbr" event={"ID":"8d80093e-2150-45d5-b78a-a96386daf7e9","Type":"ContainerStarted","Data":"9909b774e96a0ddb8f6866a4eab8545c77fbda8c45475193b7bdf86f4b10f9e4"} Jul 21 13:44:35.756971 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.756886 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-glfqj" event={"ID":"fe984719-83dd-497e-8f59-9fd42fabbde3","Type":"ContainerStarted","Data":"7e8a546e9cb7df4f38e42e7b2cc16445398fe7907ac46e2d8b26980281aa3b1d"} Jul 21 13:44:35.759057 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.758971 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" event={"ID":"80d7f298-9397-475b-951d-baf26856c339","Type":"ContainerStarted","Data":"ce283e31a6e2a8ae7de2e85ce51bb4f5802749a58262a877816ec8e4f8b4e718"} Jul 21 13:44:35.823333 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.823215 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:35.930337 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.930281 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:35.975591 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.975551 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:35.975813 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.975624 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-m57g8\" (UID: \"9ea20305-07d4-404e-91f7-fe2c5ef68598\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:44:35.975813 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.975653 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:35.975813 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.975739 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:35.975813 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.975774 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:35.975813 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.975807 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kfq8l\" (UID: \"6b173759-de6d-43c8-adaa-41a0df593ed2\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:44:35.976083 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:35.975886 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:35.976083 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.976076 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 13:44:35.976191 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.976144 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls podName:37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a nodeName:}" failed. No retries permitted until 2026-07-21 13:44:37.976124616 +0000 UTC m=+39.127965720 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ng6jq" (UID: "37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a") : secret "cluster-monitoring-operator-tls" not found Jul 21 13:44:35.976633 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.976612 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle podName:d29c3f6d-9285-48fa-979b-b71b91753fc1 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:37.976594722 +0000 UTC m=+39.128435822 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle") pod "router-default-5565c7c56b-9stjh" (UID: "d29c3f6d-9285-48fa-979b-b71b91753fc1") : configmap references non-existent config key: service-ca.crt Jul 21 13:44:35.976742 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.976718 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 13:44:35.976797 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.976757 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls podName:9ea20305-07d4-404e-91f7-fe2c5ef68598 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:37.976746273 +0000 UTC m=+39.128587374 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-m57g8" (UID: "9ea20305-07d4-404e-91f7-fe2c5ef68598") : secret "samples-operator-tls" not found Jul 21 13:44:35.976870 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.976827 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 13:44:35.976870 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.976838 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-789fb75d98-pqr2q: secret "image-registry-tls" not found Jul 21 13:44:35.976972 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.976926 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls podName:3b4040af-d0ae-46c2-ab55-880e8f31af17 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:37.976913423 +0000 UTC m=+39.128754524 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls") pod "image-registry-789fb75d98-pqr2q" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17") : secret "image-registry-tls" not found Jul 21 13:44:35.977030 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.976999 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 13:44:35.977030 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.977030 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls podName:53a477e4-4535-4679-acbb-197a47d3e114 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:37.977020476 +0000 UTC m=+39.128861578 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls") pod "dns-default-dtmfw" (UID: "53a477e4-4535-4679-acbb-197a47d3e114") : secret "dns-default-metrics-tls" not found Jul 21 13:44:35.977255 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.977081 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 13:44:35.977255 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.977110 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs podName:d29c3f6d-9285-48fa-979b-b71b91753fc1 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:37.977100519 +0000 UTC m=+39.128941621 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs") pod "router-default-5565c7c56b-9stjh" (UID: "d29c3f6d-9285-48fa-979b-b71b91753fc1") : secret "router-metrics-certs-default" not found Jul 21 13:44:35.977255 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.977160 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 13:44:35.977255 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:35.977199 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert podName:6b173759-de6d-43c8-adaa-41a0df593ed2 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:37.977180694 +0000 UTC m=+39.129021796 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-kfq8l" (UID: "6b173759-de6d-43c8-adaa-41a0df593ed2") : secret "networking-console-plugin-cert" not found Jul 21 13:44:36.028767 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:36.028594 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:36.770778 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:36.769886 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f2v6c" event={"ID":"c7b2425c-081b-45a4-86e2-a0445311d69c","Type":"ContainerStarted","Data":"85d76d9f408ebe1d0fc8a66ecd6c0676ed4d384959a61745ab34ec328c266dc5"} Jul 21 13:44:36.790151 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:36.790087 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f2v6c" podStartSLOduration=7.003996227 podStartE2EDuration="8.790062224s" totalImagesPullingTime="1.786065997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:36.789792122 +0000 UTC m=+37.941633246" watchObservedRunningTime="2026-07-21 13:44:36.790062224 +0000 UTC m=+37.941903336" Jul 21 13:44:36.795586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:36.795537 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:36.863413 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:36.863300 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:36.926296 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:36.926121 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:37.667046 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:37.666869 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:37.999108 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:37.998997 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:37.999108 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:37.999067 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:37.999108 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:37.999103 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kfq8l\" (UID: \"6b173759-de6d-43c8-adaa-41a0df593ed2\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:44:37.999683 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:37.999187 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:37.999683 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999206 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 13:44:37.999683 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:37.999228 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:37.999683 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:37.999262 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-m57g8\" (UID: \"9ea20305-07d4-404e-91f7-fe2c5ef68598\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:44:37.999683 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999282 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 13:44:37.999683 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999305 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls podName:53a477e4-4535-4679-acbb-197a47d3e114 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:41.999269202 +0000 UTC m=+43.151110292 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls") pod "dns-default-dtmfw" (UID: "53a477e4-4535-4679-acbb-197a47d3e114") : secret "dns-default-metrics-tls" not found Jul 21 13:44:37.999683 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999352 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 13:44:37.999683 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999362 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs podName:d29c3f6d-9285-48fa-979b-b71b91753fc1 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:41.999342732 +0000 UTC m=+43.151183834 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs") pod "router-default-5565c7c56b-9stjh" (UID: "d29c3f6d-9285-48fa-979b-b71b91753fc1") : secret "router-metrics-certs-default" not found Jul 21 13:44:37.999683 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999391 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 13:44:37.999683 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999415 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 13:44:37.999683 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999396 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls podName:9ea20305-07d4-404e-91f7-fe2c5ef68598 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:41.999385981 +0000 UTC m=+43.151227083 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-m57g8" (UID: "9ea20305-07d4-404e-91f7-fe2c5ef68598") : secret "samples-operator-tls" not found Jul 21 13:44:37.999683 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:37.999451 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:37.999683 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999471 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls podName:37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a nodeName:}" failed. No retries permitted until 2026-07-21 13:44:41.999450694 +0000 UTC m=+43.151291799 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ng6jq" (UID: "37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a") : secret "cluster-monitoring-operator-tls" not found Jul 21 13:44:38.000241 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999495 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert podName:6b173759-de6d-43c8-adaa-41a0df593ed2 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:41.999484703 +0000 UTC m=+43.151325794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-kfq8l" (UID: "6b173759-de6d-43c8-adaa-41a0df593ed2") : secret "networking-console-plugin-cert" not found Jul 21 13:44:38.000241 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999497 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 13:44:38.000241 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999514 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-789fb75d98-pqr2q: secret "image-registry-tls" not found Jul 21 13:44:38.000241 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999514 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle podName:d29c3f6d-9285-48fa-979b-b71b91753fc1 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:41.999506074 +0000 UTC m=+43.151347192 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle") pod "router-default-5565c7c56b-9stjh" (UID: "d29c3f6d-9285-48fa-979b-b71b91753fc1") : configmap references non-existent config key: service-ca.crt Jul 21 13:44:38.000241 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:37.999555 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls podName:3b4040af-d0ae-46c2-ab55-880e8f31af17 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:41.999548258 +0000 UTC m=+43.151389345 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls") pod "image-registry-789fb75d98-pqr2q" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17") : secret "image-registry-tls" not found Jul 21 13:44:40.367455 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:40.367249 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:40.465015 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:40.464942 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:42.040159 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:42.040114 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:42.040172 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:42.040209 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-m57g8\" (UID: \"9ea20305-07d4-404e-91f7-fe2c5ef68598\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:42.040240 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040297 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:42.040307 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040325 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040349 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle podName:d29c3f6d-9285-48fa-979b-b71b91753fc1 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:50.040330058 +0000 UTC m=+51.192171148 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle") pod "router-default-5565c7c56b-9stjh" (UID: "d29c3f6d-9285-48fa-979b-b71b91753fc1") : configmap references non-existent config key: service-ca.crt Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040379 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040379 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls podName:9ea20305-07d4-404e-91f7-fe2c5ef68598 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:50.040363807 +0000 UTC m=+51.192204896 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-m57g8" (UID: "9ea20305-07d4-404e-91f7-fe2c5ef68598") : secret "samples-operator-tls" not found Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:42.040428 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040434 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040451 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-789fb75d98-pqr2q: secret "image-registry-tls" not found Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:42.040457 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kfq8l\" (UID: \"6b173759-de6d-43c8-adaa-41a0df593ed2\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:44:42.040596 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040482 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls podName:37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a nodeName:}" failed. No retries permitted until 2026-07-21 13:44:50.040466841 +0000 UTC m=+51.192307928 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ng6jq" (UID: "37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a") : secret "cluster-monitoring-operator-tls" not found Jul 21 13:44:42.041200 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040508 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 13:44:42.041200 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040515 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 13:44:42.041200 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040511 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls podName:53a477e4-4535-4679-acbb-197a47d3e114 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:50.040498378 +0000 UTC m=+51.192339466 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls") pod "dns-default-dtmfw" (UID: "53a477e4-4535-4679-acbb-197a47d3e114") : secret "dns-default-metrics-tls" not found Jul 21 13:44:42.041200 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040550 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls podName:3b4040af-d0ae-46c2-ab55-880e8f31af17 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:50.040542178 +0000 UTC m=+51.192383268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls") pod "image-registry-789fb75d98-pqr2q" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17") : secret "image-registry-tls" not found Jul 21 13:44:42.041200 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040563 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs podName:d29c3f6d-9285-48fa-979b-b71b91753fc1 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:50.040557532 +0000 UTC m=+51.192398618 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs") pod "router-default-5565c7c56b-9stjh" (UID: "d29c3f6d-9285-48fa-979b-b71b91753fc1") : secret "router-metrics-certs-default" not found Jul 21 13:44:42.041200 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:42.040574 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert podName:6b173759-de6d-43c8-adaa-41a0df593ed2 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:50.040569103 +0000 UTC m=+51.192410190 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-kfq8l" (UID: "6b173759-de6d-43c8-adaa-41a0df593ed2") : secret "networking-console-plugin-cert" not found Jul 21 13:44:44.803434 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.801623 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" event={"ID":"cd83129f-d2e9-4efc-99be-4bc5af88e534","Type":"ContainerStarted","Data":"814b6c810315b19889048bb665839804a3f8af9dcff87301534423f745f21abe"} Jul 21 13:44:44.803434 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.801836 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:44.804289 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.804263 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p8hzg" event={"ID":"a4ff8e96-9fe5-4f87-8f9c-a2d740589863","Type":"ContainerStarted","Data":"26b329a7e9d8833bdc0fc3ab2b9b9fcf21878357da214ae95046b64156094a10"} Jul 21 13:44:44.804565 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.804445 2567 patch_prober.go:28] interesting pod/console-operator-565b7b569d-pdjqz container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused" start-of-body= Jul 21 13:44:44.804565 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.804498 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" podUID="cd83129f-d2e9-4efc-99be-4bc5af88e534" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused" Jul 21 13:44:44.812310 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.811676 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-65wbr" event={"ID":"8d80093e-2150-45d5-b78a-a96386daf7e9","Type":"ContainerStarted","Data":"bf4bd18e79d8d2323db048ea9651bb1ece0c153fd6c877f5fadafedfba3d3ed4"} Jul 21 13:44:44.816411 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.816332 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" event={"ID":"80d7f298-9397-475b-951d-baf26856c339","Type":"ContainerStarted","Data":"c22eb3c92d99b62a3a74d2ab0f3daec5ad58bb7a104614c1570fd75c19ce7844"} Jul 21 13:44:44.818618 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.818140 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" event={"ID":"108a7342-129a-4511-bc08-2b15d4407afa","Type":"ContainerStarted","Data":"0304df1e3d1e9a1056e79f4b3e38d62411a3a2c39eafb01307bd28787cc80c03"} Jul 21 13:44:44.829479 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.829307 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" podStartSLOduration=7.1797422300000004 podStartE2EDuration="16.829284678s" totalImagesPullingTime="9.649542448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:44.8279629 +0000 UTC m=+45.979804013" watchObservedRunningTime="2026-07-21 13:44:44.829284678 +0000 UTC m=+45.981125788" Jul 21 13:44:44.843603 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.843523 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:44.851442 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.849087 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" podStartSLOduration=7.124121458 podStartE2EDuration="16.849062372s" totalImagesPullingTime="9.724940914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:44.847455324 +0000 UTC m=+45.999296435" watchObservedRunningTime="2026-07-21 13:44:44.849062372 +0000 UTC m=+46.000903464" Jul 21 13:44:44.910144 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.908900 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-65wbr" podStartSLOduration=7.177653678 podStartE2EDuration="16.908876579s" totalImagesPullingTime="9.731222901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:44.908170969 +0000 UTC m=+46.060012079" watchObservedRunningTime="2026-07-21 13:44:44.908876579 +0000 UTC m=+46.060717690" Jul 21 13:44:44.910144 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.909671 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" podStartSLOduration=7.217951517 podStartE2EDuration="16.909658784s" totalImagesPullingTime="9.691707267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:44.870660851 +0000 UTC m=+46.022501960" watchObservedRunningTime="2026-07-21 13:44:44.909658784 +0000 UTC m=+46.061499895" Jul 21 13:44:44.921253 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.921092 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:44.934523 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:44.934361 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p8hzg" podStartSLOduration=1.315959345 podStartE2EDuration="10.934317598s" totalImagesPullingTime="9.618358253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:44.931423121 +0000 UTC m=+46.083264232" watchObservedRunningTime="2026-07-21 13:44:44.934317598 +0000 UTC m=+46.086158707" Jul 21 13:44:45.748818 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:45.748777 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:45.815266 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:45.815164 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:45.823984 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:45.823954 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/0.log" Jul 21 13:44:45.824181 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:45.824003 2567 generic.go:356] "Generic (PLEG): container finished" podID="cd83129f-d2e9-4efc-99be-4bc5af88e534" containerID="814b6c810315b19889048bb665839804a3f8af9dcff87301534423f745f21abe" exitCode=255 Jul 21 13:44:45.824181 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:45.824112 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" event={"ID":"cd83129f-d2e9-4efc-99be-4bc5af88e534","Type":"ContainerDied","Data":"814b6c810315b19889048bb665839804a3f8af9dcff87301534423f745f21abe"} Jul 21 13:44:45.824490 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:45.824430 2567 scope.go:117] "RemoveContainer" containerID="814b6c810315b19889048bb665839804a3f8af9dcff87301534423f745f21abe" Jul 21 13:44:45.827889 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:45.827854 2567 generic.go:356] "Generic (PLEG): container finished" podID="c2a5ace1-fdd3-4841-992a-a0f34c621d83" containerID="ddc20294bfdf56b52264047a37c159039e98c4b9cdd2e51eca7248e3af13e6d2" exitCode=0 Jul 21 13:44:45.828064 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:45.827896 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t26mf" event={"ID":"c2a5ace1-fdd3-4841-992a-a0f34c621d83","Type":"ContainerDied","Data":"ddc20294bfdf56b52264047a37c159039e98c4b9cdd2e51eca7248e3af13e6d2"} Jul 21 13:44:45.830244 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:45.830198 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-glfqj" event={"ID":"fe984719-83dd-497e-8f59-9fd42fabbde3","Type":"ContainerStarted","Data":"f612f8dedde91fc370435fde1ab86e31f8d5ed4837e695a37dbcf211a1104b54"} Jul 21 13:44:45.871048 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:45.870986 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-glfqj" podStartSLOduration=8.239633909 podStartE2EDuration="17.870960965s" totalImagesPullingTime="9.631327056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:45.869816042 +0000 UTC m=+47.021657151" watchObservedRunningTime="2026-07-21 13:44:45.870960965 +0000 UTC m=+47.022802072" Jul 21 13:44:46.053414 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.051864 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v7j4w"] Jul 21 13:44:46.069858 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.069760 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v7j4w"] Jul 21 13:44:46.070039 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.069890 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.073866 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.073840 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 13:44:46.073866 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.073853 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s5m7b\"" Jul 21 13:44:46.073866 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.073839 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 13:44:46.085793 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.085681 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c52474d7-8c91-4256-bd47-fd1aef601222-crio-socket\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.085996 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.085900 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c52474d7-8c91-4256-bd47-fd1aef601222-data-volume\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.085996 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.085933 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.086116 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.086081 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.086116 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.086113 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svpn6\" (UniqueName: \"kubernetes.io/projected/c52474d7-8c91-4256-bd47-fd1aef601222-kube-api-access-svpn6\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.101331 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.101294 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:46.186822 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.186781 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.186822 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.186824 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svpn6\" (UniqueName: \"kubernetes.io/projected/c52474d7-8c91-4256-bd47-fd1aef601222-kube-api-access-svpn6\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.187110 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:46.186960 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 13:44:46.187110 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:46.187045 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls podName:c52474d7-8c91-4256-bd47-fd1aef601222 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:46.68702145 +0000 UTC m=+47.838862549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v7j4w" (UID: "c52474d7-8c91-4256-bd47-fd1aef601222") : secret "insights-runtime-extractor-tls" not found Jul 21 13:44:46.187110 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.187073 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c52474d7-8c91-4256-bd47-fd1aef601222-crio-socket\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.187270 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.187174 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c52474d7-8c91-4256-bd47-fd1aef601222-data-volume\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.187270 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.187205 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.187375 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.187289 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c52474d7-8c91-4256-bd47-fd1aef601222-crio-socket\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.187375 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:46.187330 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 13:44:46.187457 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:46.187385 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm podName:c52474d7-8c91-4256-bd47-fd1aef601222 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:46.687369729 +0000 UTC m=+47.839210830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v7j4w" (UID: "c52474d7-8c91-4256-bd47-fd1aef601222") : configmap "kube-rbac-proxy" not found Jul 21 13:44:46.187533 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.187514 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c52474d7-8c91-4256-bd47-fd1aef601222-data-volume\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.199367 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.199315 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svpn6\" (UniqueName: \"kubernetes.io/projected/c52474d7-8c91-4256-bd47-fd1aef601222-kube-api-access-svpn6\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.230448 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.230402 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:46.330182 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.330065 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:46.447429 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.447388 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:46.547361 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.547307 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:46.692681 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.692638 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.692890 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.692772 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:46.692890 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:46.692785 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 13:44:46.692890 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:46.692867 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 13:44:46.693001 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:46.692871 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm podName:c52474d7-8c91-4256-bd47-fd1aef601222 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:47.692854874 +0000 UTC m=+48.844695962 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v7j4w" (UID: "c52474d7-8c91-4256-bd47-fd1aef601222") : configmap "kube-rbac-proxy" not found Jul 21 13:44:46.693001 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:46.692917 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls podName:c52474d7-8c91-4256-bd47-fd1aef601222 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:47.692905342 +0000 UTC m=+48.844746429 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v7j4w" (UID: "c52474d7-8c91-4256-bd47-fd1aef601222") : secret "insights-runtime-extractor-tls" not found Jul 21 13:44:46.836515 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.836485 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 13:44:46.836986 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.836966 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/0.log" Jul 21 13:44:46.837046 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.837006 2567 generic.go:356] "Generic (PLEG): container finished" podID="cd83129f-d2e9-4efc-99be-4bc5af88e534" containerID="f9f6c14fde8f5af7123c1ba8e02303fd1eb309dfd27cb246a1dd1b393f207355" exitCode=255 Jul 21 13:44:46.837107 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.837043 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" event={"ID":"cd83129f-d2e9-4efc-99be-4bc5af88e534","Type":"ContainerDied","Data":"f9f6c14fde8f5af7123c1ba8e02303fd1eb309dfd27cb246a1dd1b393f207355"} Jul 21 13:44:46.837157 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.837131 2567 scope.go:117] "RemoveContainer" containerID="814b6c810315b19889048bb665839804a3f8af9dcff87301534423f745f21abe" Jul 21 13:44:46.837461 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.837441 2567 scope.go:117] "RemoveContainer" containerID="f9f6c14fde8f5af7123c1ba8e02303fd1eb309dfd27cb246a1dd1b393f207355" Jul 21 13:44:46.837774 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:46.837749 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-pdjqz_openshift-console-operator(cd83129f-d2e9-4efc-99be-4bc5af88e534)\"" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" podUID="cd83129f-d2e9-4efc-99be-4bc5af88e534" Jul 21 13:44:46.840573 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.840538 2567 generic.go:356] "Generic (PLEG): container finished" podID="c2a5ace1-fdd3-4841-992a-a0f34c621d83" containerID="0e276ec69d021765f2b1955ee75482bb07378da5d42a48b2a8167756695ba533" exitCode=0 Jul 21 13:44:46.840749 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:46.840662 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t26mf" event={"ID":"c2a5ace1-fdd3-4841-992a-a0f34c621d83","Type":"ContainerDied","Data":"0e276ec69d021765f2b1955ee75482bb07378da5d42a48b2a8167756695ba533"} Jul 21 13:44:47.136780 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.136743 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-d4t5n"] Jul 21 13:44:47.151192 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.151161 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-d4t5n"] Jul 21 13:44:47.151381 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.151280 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-d4t5n" Jul 21 13:44:47.154262 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.154221 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 21 13:44:47.154262 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.154250 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-tcqzz\"" Jul 21 13:44:47.154504 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.154230 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 21 13:44:47.181402 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.181359 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:47.198608 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.198572 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm764\" (UniqueName: \"kubernetes.io/projected/73189f81-d993-4346-9fc5-be10ee914c94-kube-api-access-zm764\") pod \"migrator-7cc9c7ffd7-d4t5n\" (UID: \"73189f81-d993-4346-9fc5-be10ee914c94\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-d4t5n" Jul 21 13:44:47.282746 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.282614 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:47.299786 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.299727 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm764\" (UniqueName: \"kubernetes.io/projected/73189f81-d993-4346-9fc5-be10ee914c94-kube-api-access-zm764\") pod \"migrator-7cc9c7ffd7-d4t5n\" (UID: \"73189f81-d993-4346-9fc5-be10ee914c94\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-d4t5n" Jul 21 13:44:47.311685 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.311654 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm764\" (UniqueName: \"kubernetes.io/projected/73189f81-d993-4346-9fc5-be10ee914c94-kube-api-access-zm764\") pod \"migrator-7cc9c7ffd7-d4t5n\" (UID: \"73189f81-d993-4346-9fc5-be10ee914c94\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-d4t5n" Jul 21 13:44:47.455088 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.455052 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-js4ds_2b19a38a-ffc0-4c33-914c-0eb1b2c67b71/dns-node-resolver/0.log" Jul 21 13:44:47.462213 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.462179 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-d4t5n" Jul 21 13:44:47.632777 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.632743 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-d4t5n"] Jul 21 13:44:47.635746 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:47.635689 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73189f81_d993_4346_9fc5_be10ee914c94.slice/crio-560ce600a722e8bc42adba10d9ff57e3054a501b450c6e7864d2bfb085fdae08 WatchSource:0}: Error finding container 560ce600a722e8bc42adba10d9ff57e3054a501b450c6e7864d2bfb085fdae08: Status 404 returned error can't find the container with id 560ce600a722e8bc42adba10d9ff57e3054a501b450c6e7864d2bfb085fdae08 Jul 21 13:44:47.704567 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.704523 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:47.704789 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.704648 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:47.704789 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:47.704722 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 13:44:47.704789 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:47.704778 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 13:44:47.704900 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:47.704796 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls podName:c52474d7-8c91-4256-bd47-fd1aef601222 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:49.704779302 +0000 UTC m=+50.856620389 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v7j4w" (UID: "c52474d7-8c91-4256-bd47-fd1aef601222") : secret "insights-runtime-extractor-tls" not found Jul 21 13:44:47.704900 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:47.704828 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm podName:c52474d7-8c91-4256-bd47-fd1aef601222 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:49.704815 +0000 UTC m=+50.856656087 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v7j4w" (UID: "c52474d7-8c91-4256-bd47-fd1aef601222") : configmap "kube-rbac-proxy" not found Jul 21 13:44:47.845497 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.845392 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-d4t5n" event={"ID":"73189f81-d993-4346-9fc5-be10ee914c94","Type":"ContainerStarted","Data":"560ce600a722e8bc42adba10d9ff57e3054a501b450c6e7864d2bfb085fdae08"} Jul 21 13:44:47.846993 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.846969 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 13:44:47.847352 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.847325 2567 scope.go:117] "RemoveContainer" containerID="f9f6c14fde8f5af7123c1ba8e02303fd1eb309dfd27cb246a1dd1b393f207355" Jul 21 13:44:47.847534 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:47.847509 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-pdjqz_openshift-console-operator(cd83129f-d2e9-4efc-99be-4bc5af88e534)\"" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" podUID="cd83129f-d2e9-4efc-99be-4bc5af88e534" Jul 21 13:44:47.850670 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.850637 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t26mf" event={"ID":"c2a5ace1-fdd3-4841-992a-a0f34c621d83","Type":"ContainerStarted","Data":"74d3e77b8567e31391c4d85716918ac35fbc70007391c66cb9aa2c985d9876a2"} Jul 21 13:44:47.897505 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:47.897441 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-t26mf" podStartSLOduration=8.883749039 podStartE2EDuration="47.897423199s" totalImagesPullingTime="39.01367416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:47.895878882 +0000 UTC m=+49.047719990" watchObservedRunningTime="2026-07-21 13:44:47.897423199 +0000 UTC m=+49.049264337" Jul 21 13:44:48.033169 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.033124 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:48.063530 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.063477 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-qtmmj"] Jul 21 13:44:48.066048 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.066000 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bbnk6_f26b564f-04fd-4e02-ade5-45adcf879547/node-ca/0.log" Jul 21 13:44:48.101075 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.100894 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-qtmmj"] Jul 21 13:44:48.101382 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.101170 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-qtmmj" Jul 21 13:44:48.104827 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.104755 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 21 13:44:48.104827 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.104755 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-fxd8w\"" Jul 21 13:44:48.104827 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.104818 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 21 13:44:48.104827 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.104840 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 21 13:44:48.105328 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.104755 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 21 13:44:48.122409 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.122342 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:48.210995 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.210949 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/593bcd4f-fbb4-4bb0-a81b-2c0c9319480f-signing-cabundle\") pod \"service-ca-58598695f7-qtmmj\" (UID: \"593bcd4f-fbb4-4bb0-a81b-2c0c9319480f\") " pod="openshift-service-ca/service-ca-58598695f7-qtmmj" Jul 21 13:44:48.211177 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.211026 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfz9w\" (UniqueName: \"kubernetes.io/projected/593bcd4f-fbb4-4bb0-a81b-2c0c9319480f-kube-api-access-jfz9w\") pod \"service-ca-58598695f7-qtmmj\" (UID: \"593bcd4f-fbb4-4bb0-a81b-2c0c9319480f\") " pod="openshift-service-ca/service-ca-58598695f7-qtmmj" Jul 21 13:44:48.211177 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.211166 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/593bcd4f-fbb4-4bb0-a81b-2c0c9319480f-signing-key\") pod \"service-ca-58598695f7-qtmmj\" (UID: \"593bcd4f-fbb4-4bb0-a81b-2c0c9319480f\") " pod="openshift-service-ca/service-ca-58598695f7-qtmmj" Jul 21 13:44:48.312778 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.312726 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/593bcd4f-fbb4-4bb0-a81b-2c0c9319480f-signing-cabundle\") pod \"service-ca-58598695f7-qtmmj\" (UID: \"593bcd4f-fbb4-4bb0-a81b-2c0c9319480f\") " pod="openshift-service-ca/service-ca-58598695f7-qtmmj" Jul 21 13:44:48.312959 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.312854 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfz9w\" (UniqueName: \"kubernetes.io/projected/593bcd4f-fbb4-4bb0-a81b-2c0c9319480f-kube-api-access-jfz9w\") pod \"service-ca-58598695f7-qtmmj\" (UID: \"593bcd4f-fbb4-4bb0-a81b-2c0c9319480f\") " pod="openshift-service-ca/service-ca-58598695f7-qtmmj" Jul 21 13:44:48.313039 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.312969 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/593bcd4f-fbb4-4bb0-a81b-2c0c9319480f-signing-key\") pod \"service-ca-58598695f7-qtmmj\" (UID: \"593bcd4f-fbb4-4bb0-a81b-2c0c9319480f\") " pod="openshift-service-ca/service-ca-58598695f7-qtmmj" Jul 21 13:44:48.313490 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.313448 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/593bcd4f-fbb4-4bb0-a81b-2c0c9319480f-signing-cabundle\") pod \"service-ca-58598695f7-qtmmj\" (UID: \"593bcd4f-fbb4-4bb0-a81b-2c0c9319480f\") " pod="openshift-service-ca/service-ca-58598695f7-qtmmj" Jul 21 13:44:48.315413 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.315384 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/593bcd4f-fbb4-4bb0-a81b-2c0c9319480f-signing-key\") pod \"service-ca-58598695f7-qtmmj\" (UID: \"593bcd4f-fbb4-4bb0-a81b-2c0c9319480f\") " pod="openshift-service-ca/service-ca-58598695f7-qtmmj" Jul 21 13:44:48.323398 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.323369 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfz9w\" (UniqueName: \"kubernetes.io/projected/593bcd4f-fbb4-4bb0-a81b-2c0c9319480f-kube-api-access-jfz9w\") pod \"service-ca-58598695f7-qtmmj\" (UID: \"593bcd4f-fbb4-4bb0-a81b-2c0c9319480f\") " pod="openshift-service-ca/service-ca-58598695f7-qtmmj" Jul 21 13:44:48.418552 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.418512 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-qtmmj" Jul 21 13:44:48.585311 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.585274 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-qtmmj"] Jul 21 13:44:48.588245 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:44:48.588201 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod593bcd4f_fbb4_4bb0_a81b_2c0c9319480f.slice/crio-8a75e51875ee48e1aa0d172bf1e17e983cd74086808fafe21809da1d390a331c WatchSource:0}: Error finding container 8a75e51875ee48e1aa0d172bf1e17e983cd74086808fafe21809da1d390a331c: Status 404 returned error can't find the container with id 8a75e51875ee48e1aa0d172bf1e17e983cd74086808fafe21809da1d390a331c Jul 21 13:44:48.608520 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.608483 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:48.712629 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.712596 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:48.856711 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.856670 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-qtmmj" event={"ID":"593bcd4f-fbb4-4bb0-a81b-2c0c9319480f","Type":"ContainerStarted","Data":"b58273e83f75c4f1f0378b958877bfb84aa95f51d25102d5358d2b1789148661"} Jul 21 13:44:48.857193 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.856750 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-qtmmj" event={"ID":"593bcd4f-fbb4-4bb0-a81b-2c0c9319480f","Type":"ContainerStarted","Data":"8a75e51875ee48e1aa0d172bf1e17e983cd74086808fafe21809da1d390a331c"} Jul 21 13:44:48.857193 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.856998 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p8hzg_a4ff8e96-9fe5-4f87-8f9c-a2d740589863/serve-healthcheck-canary/0.log" Jul 21 13:44:48.878368 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.878312 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-qtmmj" podStartSLOduration=0.878294202 podStartE2EDuration="878.294202ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:48.878045083 +0000 UTC m=+50.029886193" watchObservedRunningTime="2026-07-21 13:44:48.878294202 +0000 UTC m=+50.030135305" Jul 21 13:44:48.955304 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:48.955199 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:49.726638 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:49.726591 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:49.726835 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:49.726755 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 13:44:49.726835 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:49.726800 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:49.726938 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:49.726838 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm podName:c52474d7-8c91-4256-bd47-fd1aef601222 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:53.72681614 +0000 UTC m=+54.878657246 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v7j4w" (UID: "c52474d7-8c91-4256-bd47-fd1aef601222") : configmap "kube-rbac-proxy" not found Jul 21 13:44:49.726938 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:49.726911 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 13:44:49.727046 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:49.726971 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls podName:c52474d7-8c91-4256-bd47-fd1aef601222 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:53.726951924 +0000 UTC m=+54.878793014 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v7j4w" (UID: "c52474d7-8c91-4256-bd47-fd1aef601222") : secret "insights-runtime-extractor-tls" not found Jul 21 13:44:50.131494 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.131446 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:44:50.132060 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.131517 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:50.132060 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.131556 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-m57g8\" (UID: \"9ea20305-07d4-404e-91f7-fe2c5ef68598\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:44:50.132060 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.131588 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:44:50.132060 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.131641 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 13:44:50.132060 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.131663 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:44:50.132060 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.131709 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:44:50.132060 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.131757 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls podName:37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a nodeName:}" failed. No retries permitted until 2026-07-21 13:45:06.131735695 +0000 UTC m=+67.283576787 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ng6jq" (UID: "37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a") : secret "cluster-monitoring-operator-tls" not found Jul 21 13:44:50.132060 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.131778 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kfq8l\" (UID: \"6b173759-de6d-43c8-adaa-41a0df593ed2\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:44:50.132060 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.131930 2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 13:44:50.132060 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.131983 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert podName:6b173759-de6d-43c8-adaa-41a0df593ed2 nodeName:}" failed. No retries permitted until 2026-07-21 13:45:06.1319683 +0000 UTC m=+67.283809392 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-kfq8l" (UID: "6b173759-de6d-43c8-adaa-41a0df593ed2") : secret "networking-console-plugin-cert" not found Jul 21 13:44:50.132060 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.132052 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle podName:d29c3f6d-9285-48fa-979b-b71b91753fc1 nodeName:}" failed. No retries permitted until 2026-07-21 13:45:06.132041557 +0000 UTC m=+67.283882650 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle") pod "router-default-5565c7c56b-9stjh" (UID: "d29c3f6d-9285-48fa-979b-b71b91753fc1") : configmap references non-existent config key: service-ca.crt Jul 21 13:44:50.132634 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.132112 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 13:44:50.132634 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.132148 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls podName:9ea20305-07d4-404e-91f7-fe2c5ef68598 nodeName:}" failed. No retries permitted until 2026-07-21 13:45:06.132134154 +0000 UTC m=+67.283975245 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-m57g8" (UID: "9ea20305-07d4-404e-91f7-fe2c5ef68598") : secret "samples-operator-tls" not found Jul 21 13:44:50.132634 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.132170 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 13:44:50.132634 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.132202 2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 13:44:50.132634 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.132219 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls podName:53a477e4-4535-4679-acbb-197a47d3e114 nodeName:}" failed. No retries permitted until 2026-07-21 13:45:06.13219973 +0000 UTC m=+67.284040821 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls") pod "dns-default-dtmfw" (UID: "53a477e4-4535-4679-acbb-197a47d3e114") : secret "dns-default-metrics-tls" not found Jul 21 13:44:50.132634 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.132236 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs podName:d29c3f6d-9285-48fa-979b-b71b91753fc1 nodeName:}" failed. No retries permitted until 2026-07-21 13:45:06.132227202 +0000 UTC m=+67.284068297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs") pod "router-default-5565c7c56b-9stjh" (UID: "d29c3f6d-9285-48fa-979b-b71b91753fc1") : secret "router-metrics-certs-default" not found Jul 21 13:44:50.132634 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.132289 2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 13:44:50.132634 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.132301 2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-789fb75d98-pqr2q: secret "image-registry-tls" not found Jul 21 13:44:50.132634 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:50.132351 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls podName:3b4040af-d0ae-46c2-ab55-880e8f31af17 nodeName:}" failed. No retries permitted until 2026-07-21 13:45:06.132338955 +0000 UTC m=+67.284180045 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls") pod "image-registry-789fb75d98-pqr2q" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17") : secret "image-registry-tls" not found Jul 21 13:44:50.186369 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.186325 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:50.458951 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.458911 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:50.560964 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.560858 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:50.715845 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.715806 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:50.867749 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.867633 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-d4t5n" event={"ID":"73189f81-d993-4346-9fc5-be10ee914c94","Type":"ContainerStarted","Data":"029f62000a5f96c1ff0841566d47546bde043128cfd78267c0003a2d27695f59"} Jul 21 13:44:50.867749 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.867721 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-d4t5n" event={"ID":"73189f81-d993-4346-9fc5-be10ee914c94","Type":"ContainerStarted","Data":"720169bf64f080331d05ba46f500c465e9d5d728fd9f9ebbc8ab3c398a2ff90d"} Jul 21 13:44:50.915157 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.915091 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-d4t5n" podStartSLOduration=1.524866316 podStartE2EDuration="3.915066826s" totalImagesPullingTime="2.39020051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:50.913566262 +0000 UTC m=+52.065407386" watchObservedRunningTime="2026-07-21 13:44:50.915066826 +0000 UTC m=+52.066907936" Jul 21 13:44:50.927537 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:50.927499 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:51.565284 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:51.565246 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:51.698184 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:51.698141 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:52.250246 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:52.250206 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:52.448856 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:52.448817 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:53.505853 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:53.505803 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:53.579912 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:53.579877 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:53.672007 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:53.671963 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:53.771606 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:53.771500 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:53.771829 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:53.771639 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:53.771902 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:53.771818 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 13:44:53.771955 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:53.771908 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls podName:c52474d7-8c91-4256-bd47-fd1aef601222 nodeName:}" failed. No retries permitted until 2026-07-21 13:45:01.771884685 +0000 UTC m=+62.923725791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v7j4w" (UID: "c52474d7-8c91-4256-bd47-fd1aef601222") : secret "insights-runtime-extractor-tls" not found Jul 21 13:44:53.772159 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:53.772137 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c52474d7-8c91-4256-bd47-fd1aef601222-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:44:53.857534 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:53.857495 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:53.953438 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:53.953399 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:54.042728 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:54.042594 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:54.247251 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:54.247209 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:54.589493 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:54.589451 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:54.589920 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:54.589904 2567 scope.go:117] "RemoveContainer" containerID="f9f6c14fde8f5af7123c1ba8e02303fd1eb309dfd27cb246a1dd1b393f207355" Jul 21 13:44:54.590085 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:54.590060 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-pdjqz_openshift-console-operator(cd83129f-d2e9-4efc-99be-4bc5af88e534)\"" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" podUID="cd83129f-d2e9-4efc-99be-4bc5af88e534" Jul 21 13:44:54.802093 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:54.802054 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:44:54.846446 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:54.846334 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:54.882097 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:54.882065 2567 scope.go:117] "RemoveContainer" containerID="f9f6c14fde8f5af7123c1ba8e02303fd1eb309dfd27cb246a1dd1b393f207355" Jul 21 13:44:54.882298 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:44:54.882274 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-pdjqz_openshift-console-operator(cd83129f-d2e9-4efc-99be-4bc5af88e534)\"" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" podUID="cd83129f-d2e9-4efc-99be-4bc5af88e534" Jul 21 13:44:54.950999 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:54.950966 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:55.120407 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:55.120314 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:55.261447 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:55.261408 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:55.647264 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:55.647228 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:55.749757 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:55.749720 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:55.847199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:55.847147 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:55.948857 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:55.948826 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:56.249541 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:56.249433 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:56.350683 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:56.350643 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:56.838828 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:56.838785 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:57.503450 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:57.503410 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:57.649161 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:57.649124 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:57.841550 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:57.841438 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:58.241062 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:58.241026 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:58.853483 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:58.853446 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:59.253228 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:59.253188 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:59.661357 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:59.661324 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:59.839639 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:59.839595 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:44:59.940238 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:44:59.940147 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:00.063494 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:00.063455 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:00.264919 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:00.264838 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:00.651363 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:00.651323 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:00.840610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:00.840575 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:01.715764 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:01.715729 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h66bv" Jul 21 13:45:01.848840 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:01.848783 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:45:01.851137 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:01.851114 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c52474d7-8c91-4256-bd47-fd1aef601222-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v7j4w\" (UID: \"c52474d7-8c91-4256-bd47-fd1aef601222\") " pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:45:02.063932 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:02.063844 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s5m7b\"" Jul 21 13:45:02.071652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:02.071620 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v7j4w" Jul 21 13:45:02.209615 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:02.209449 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v7j4w"] Jul 21 13:45:02.212358 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:45:02.212330 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc52474d7_8c91_4256_bd47_fd1aef601222.slice/crio-6d60a64b60045f88f69e04caa4561fa441feca53a03267a05efadec12d808f73 WatchSource:0}: Error finding container 6d60a64b60045f88f69e04caa4561fa441feca53a03267a05efadec12d808f73: Status 404 returned error can't find the container with id 6d60a64b60045f88f69e04caa4561fa441feca53a03267a05efadec12d808f73 Jul 21 13:45:02.908916 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:02.908875 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v7j4w" event={"ID":"c52474d7-8c91-4256-bd47-fd1aef601222","Type":"ContainerStarted","Data":"6d60a64b60045f88f69e04caa4561fa441feca53a03267a05efadec12d808f73"} Jul 21 13:45:05.185208 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:05.185164 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhhff\" (UniqueName: \"kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff\") pod \"network-check-target-2cps2\" (UID: \"4cfe6a65-9649-46d6-a3c2-723dc4e9e218\") " pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:45:05.185629 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:05.185229 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:45:05.185629 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:05.185340 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs\") pod \"network-metrics-daemon-xlpnd\" (UID: \"d08982d1-044f-462c-a84a-d4c65f1cbbef\") " pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:45:05.188144 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:05.188113 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhhff\" (UniqueName: \"kubernetes.io/projected/4cfe6a65-9649-46d6-a3c2-723dc4e9e218-kube-api-access-xhhff\") pod \"network-check-target-2cps2\" (UID: \"4cfe6a65-9649-46d6-a3c2-723dc4e9e218\") " pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:45:05.188832 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:05.188787 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 13:45:05.188962 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:05.188787 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 13:45:05.198536 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:05.198514 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093b3a4b-ec79-4a4d-a86f-05ea62a88e2c-original-pull-secret\") pod \"global-pull-secret-syncer-xvrvl\" (UID: \"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c\") " pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:45:05.198536 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:05.198530 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d08982d1-044f-462c-a84a-d4c65f1cbbef-metrics-certs\") pod \"network-metrics-daemon-xlpnd\" (UID: \"d08982d1-044f-462c-a84a-d4c65f1cbbef\") " pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:45:05.246160 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:05.246123 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-29gkd\"" Jul 21 13:45:05.253804 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:05.253777 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:45:05.442073 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:05.441985 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvrvl" Jul 21 13:45:05.451591 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:05.451565 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pt8sb\"" Jul 21 13:45:05.458707 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:05.458674 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlpnd" Jul 21 13:45:06.194339 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.194286 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:45:06.194831 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.194354 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kfq8l\" (UID: \"6b173759-de6d-43c8-adaa-41a0df593ed2\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:45:06.194831 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.194432 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:45:06.194831 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.194483 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:45:06.194831 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.194528 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-m57g8\" (UID: \"9ea20305-07d4-404e-91f7-fe2c5ef68598\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:45:06.194831 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.194559 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:45:06.194831 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.194627 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:45:06.199497 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.199410 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ng6jq\" (UID: \"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:45:06.199497 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.199465 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29c3f6d-9285-48fa-979b-b71b91753fc1-service-ca-bundle\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:45:06.200752 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.200679 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/53a477e4-4535-4679-acbb-197a47d3e114-metrics-tls\") pod \"dns-default-dtmfw\" (UID: \"53a477e4-4535-4679-acbb-197a47d3e114\") " pod="openshift-dns/dns-default-dtmfw" Jul 21 13:45:06.201548 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.201516 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls\") pod \"image-registry-789fb75d98-pqr2q\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:45:06.201667 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.201621 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ea20305-07d4-404e-91f7-fe2c5ef68598-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-m57g8\" (UID: \"9ea20305-07d4-404e-91f7-fe2c5ef68598\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:45:06.201998 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.201966 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6b173759-de6d-43c8-adaa-41a0df593ed2-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-kfq8l\" (UID: \"6b173759-de6d-43c8-adaa-41a0df593ed2\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:45:06.202137 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.202108 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d29c3f6d-9285-48fa-979b-b71b91753fc1-metrics-certs\") pod \"router-default-5565c7c56b-9stjh\" (UID: \"d29c3f6d-9285-48fa-979b-b71b91753fc1\") " pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:45:06.324277 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.324247 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-cf9ph\"" Jul 21 13:45:06.331693 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.331661 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:45:06.349109 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.348844 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gz5rr\"" Jul 21 13:45:06.356138 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.356117 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" Jul 21 13:45:06.369321 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.369293 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mm57m\"" Jul 21 13:45:06.376781 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.376752 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:45:06.379503 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.379478 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kfz4h\"" Jul 21 13:45:06.386994 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.386957 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" Jul 21 13:45:06.393646 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.393615 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qjfhr\"" Jul 21 13:45:06.401462 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.401431 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" Jul 21 13:45:06.408652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.408627 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2pd27\"" Jul 21 13:45:06.415967 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:06.415935 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dtmfw" Jul 21 13:45:07.528404 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.527394 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l"] Jul 21 13:45:07.584020 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.583950 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5565c7c56b-9stjh"] Jul 21 13:45:07.587526 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:45:07.587498 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29c3f6d_9285_48fa_979b_b71b91753fc1.slice/crio-86bb5cabfe13a570cb1bf9cad9a1b7223a25a90b11b2c9b77e8cea2dbd6f5f45 WatchSource:0}: Error finding container 86bb5cabfe13a570cb1bf9cad9a1b7223a25a90b11b2c9b77e8cea2dbd6f5f45: Status 404 returned error can't find the container with id 86bb5cabfe13a570cb1bf9cad9a1b7223a25a90b11b2c9b77e8cea2dbd6f5f45 Jul 21 13:45:07.814802 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.814777 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dtmfw"] Jul 21 13:45:07.818336 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:45:07.817963 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a477e4_4535_4679_acbb_197a47d3e114.slice/crio-38150a65fc086e609b258628fc6a61bddc736861835e030259a725772c704410 WatchSource:0}: Error finding container 38150a65fc086e609b258628fc6a61bddc736861835e030259a725772c704410: Status 404 returned error can't find the container with id 38150a65fc086e609b258628fc6a61bddc736861835e030259a725772c704410 Jul 21 13:45:07.818610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.818573 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq"] Jul 21 13:45:07.820169 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.820074 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8"] Jul 21 13:45:07.821332 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.821270 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-789fb75d98-pqr2q"] Jul 21 13:45:07.822976 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:45:07.822925 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b4040af_d0ae_46c2_ab55_880e8f31af17.slice/crio-4e3e8bddce4d4ef5db6957f6dd6a26d95d5e5e02ca78105e7c00578be556541f WatchSource:0}: Error finding container 4e3e8bddce4d4ef5db6957f6dd6a26d95d5e5e02ca78105e7c00578be556541f: Status 404 returned error can't find the container with id 4e3e8bddce4d4ef5db6957f6dd6a26d95d5e5e02ca78105e7c00578be556541f Jul 21 13:45:07.823497 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:45:07.823472 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37b1bb9e_3013_4c9a_87b8_bc120b0a9f3a.slice/crio-8d51cb8d75389f5a35a36f29b961382379b267adeaaeabcda037a1930da204ff WatchSource:0}: Error finding container 8d51cb8d75389f5a35a36f29b961382379b267adeaaeabcda037a1930da204ff: Status 404 returned error can't find the container with id 8d51cb8d75389f5a35a36f29b961382379b267adeaaeabcda037a1930da204ff Jul 21 13:45:07.864205 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.863798 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2cps2"] Jul 21 13:45:07.865591 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.865519 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xvrvl"] Jul 21 13:45:07.866420 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.866347 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xlpnd"] Jul 21 13:45:07.869278 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:45:07.868664 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cfe6a65_9649_46d6_a3c2_723dc4e9e218.slice/crio-512435bb01f3b158bf0582edd12b0aed52550786cc50ff1319bc5d4efca202e8 WatchSource:0}: Error finding container 512435bb01f3b158bf0582edd12b0aed52550786cc50ff1319bc5d4efca202e8: Status 404 returned error can't find the container with id 512435bb01f3b158bf0582edd12b0aed52550786cc50ff1319bc5d4efca202e8 Jul 21 13:45:07.869347 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:45:07.869281 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08982d1_044f_462c_a84a_d4c65f1cbbef.slice/crio-e033f744a803aee800df11e0bcfa0a68b594ea06941a10e52f01fb6526bb852d WatchSource:0}: Error finding container e033f744a803aee800df11e0bcfa0a68b594ea06941a10e52f01fb6526bb852d: Status 404 returned error can't find the container with id e033f744a803aee800df11e0bcfa0a68b594ea06941a10e52f01fb6526bb852d Jul 21 13:45:07.869862 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:45:07.869842 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod093b3a4b_ec79_4a4d_a86f_05ea62a88e2c.slice/crio-b375e25cd8cd9a7cbe41c8791936638eef9207f92fd6f7fbbaf0ac6579435162 WatchSource:0}: Error finding container b375e25cd8cd9a7cbe41c8791936638eef9207f92fd6f7fbbaf0ac6579435162: Status 404 returned error can't find the container with id b375e25cd8cd9a7cbe41c8791936638eef9207f92fd6f7fbbaf0ac6579435162 Jul 21 13:45:07.927156 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.927124 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" event={"ID":"9ea20305-07d4-404e-91f7-fe2c5ef68598","Type":"ContainerStarted","Data":"9d2e783a4e4a44a521fcad114c1995fc41caa17106583e750b67b699a17adc9d"} Jul 21 13:45:07.928624 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.928598 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" event={"ID":"3b4040af-d0ae-46c2-ab55-880e8f31af17","Type":"ContainerStarted","Data":"82f8b3ab575ca41c2942ecb3167568a7edd118a73896a0ddfbfba8c4ea1e0e51"} Jul 21 13:45:07.928774 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.928636 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" event={"ID":"3b4040af-d0ae-46c2-ab55-880e8f31af17","Type":"ContainerStarted","Data":"4e3e8bddce4d4ef5db6957f6dd6a26d95d5e5e02ca78105e7c00578be556541f"} Jul 21 13:45:07.928774 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.928671 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:45:07.929854 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.929829 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xlpnd" event={"ID":"d08982d1-044f-462c-a84a-d4c65f1cbbef","Type":"ContainerStarted","Data":"e033f744a803aee800df11e0bcfa0a68b594ea06941a10e52f01fb6526bb852d"} Jul 21 13:45:07.931144 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.931127 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2cps2" event={"ID":"4cfe6a65-9649-46d6-a3c2-723dc4e9e218","Type":"ContainerStarted","Data":"64d7ca45353e8c6e9e9324cf8b9d7402b16484975bcfa93fb4c29834b9ed8410"} Jul 21 13:45:07.931226 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.931157 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2cps2" event={"ID":"4cfe6a65-9649-46d6-a3c2-723dc4e9e218","Type":"ContainerStarted","Data":"512435bb01f3b158bf0582edd12b0aed52550786cc50ff1319bc5d4efca202e8"} Jul 21 13:45:07.931292 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.931223 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:45:07.932668 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.932639 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5565c7c56b-9stjh" event={"ID":"d29c3f6d-9285-48fa-979b-b71b91753fc1","Type":"ContainerStarted","Data":"7c2005887c1e43fbdce781caf86d7eac891cbed80dc7e07558a8ef6c9f867513"} Jul 21 13:45:07.932778 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.932675 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5565c7c56b-9stjh" event={"ID":"d29c3f6d-9285-48fa-979b-b71b91753fc1","Type":"ContainerStarted","Data":"86bb5cabfe13a570cb1bf9cad9a1b7223a25a90b11b2c9b77e8cea2dbd6f5f45"} Jul 21 13:45:07.933738 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.933717 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dtmfw" event={"ID":"53a477e4-4535-4679-acbb-197a47d3e114","Type":"ContainerStarted","Data":"38150a65fc086e609b258628fc6a61bddc736861835e030259a725772c704410"} Jul 21 13:45:07.934711 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.934674 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" event={"ID":"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a","Type":"ContainerStarted","Data":"8d51cb8d75389f5a35a36f29b961382379b267adeaaeabcda037a1930da204ff"} Jul 21 13:45:07.935784 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.935768 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" event={"ID":"6b173759-de6d-43c8-adaa-41a0df593ed2","Type":"ContainerStarted","Data":"bca3a42d1ec90e2e4da5df11ea67ef3cfd421b8fd137306b882a96b90463b0f2"} Jul 21 13:45:07.936798 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.936780 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xvrvl" event={"ID":"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c","Type":"ContainerStarted","Data":"b375e25cd8cd9a7cbe41c8791936638eef9207f92fd6f7fbbaf0ac6579435162"} Jul 21 13:45:07.937980 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.937954 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v7j4w" event={"ID":"c52474d7-8c91-4256-bd47-fd1aef601222","Type":"ContainerStarted","Data":"070369c7397b11d09076571a177735ae5cafd7f3f9f797704aebb2e02b830da7"} Jul 21 13:45:07.949309 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.949271 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" podStartSLOduration=45.949258553 podStartE2EDuration="45.949258553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:07.948155851 +0000 UTC m=+69.099996961" watchObservedRunningTime="2026-07-21 13:45:07.949258553 +0000 UTC m=+69.101099640" Jul 21 13:45:07.967278 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.967231 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5565c7c56b-9stjh" podStartSLOduration=39.967218558 podStartE2EDuration="39.967218558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:07.965487282 +0000 UTC m=+69.117328402" watchObservedRunningTime="2026-07-21 13:45:07.967218558 +0000 UTC m=+69.119059666" Jul 21 13:45:07.984780 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:07.984720 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2cps2" podStartSLOduration=68.984680814 podStartE2EDuration="1m8.984680814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:43:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:07.983361198 +0000 UTC m=+69.135202308" watchObservedRunningTime="2026-07-21 13:45:07.984680814 +0000 UTC m=+69.136521924" Jul 21 13:45:08.332298 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:08.332265 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:45:08.335146 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:08.335121 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:45:08.944302 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:08.944236 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:45:08.946894 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:08.946667 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5565c7c56b-9stjh" Jul 21 13:45:09.530622 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:09.530200 2567 scope.go:117] "RemoveContainer" containerID="f9f6c14fde8f5af7123c1ba8e02303fd1eb309dfd27cb246a1dd1b393f207355" Jul 21 13:45:10.294546 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:10.294415 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:10.423066 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:10.422946 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:10.640105 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:10.640009 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:10.776227 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:10.776193 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:13.264293 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:13.264254 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:13.327671 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:13.327632 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:16.494748 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.494711 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:16.704062 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.703971 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:16.956671 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.956602 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:16.980348 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.980312 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dtmfw" event={"ID":"53a477e4-4535-4679-acbb-197a47d3e114","Type":"ContainerStarted","Data":"bdba2d3da8ff4cdbecbe08d7677dfac27108a8afc5453cbf33db86e6abd04027"} Jul 21 13:45:16.980658 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.980624 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dtmfw" event={"ID":"53a477e4-4535-4679-acbb-197a47d3e114","Type":"ContainerStarted","Data":"38036f5e26882fff83fe26c2dbc9d9cebbf3c1ade817f8c3790519eb0bd7b3a0"} Jul 21 13:45:16.980823 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.980808 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dtmfw" Jul 21 13:45:16.982893 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.982851 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" event={"ID":"37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a","Type":"ContainerStarted","Data":"d4e2c037fd248339fc6b0f859f69cab1d4bbeb888033c000b490e8850cbc8344"} Jul 21 13:45:16.990971 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.986182 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" event={"ID":"6b173759-de6d-43c8-adaa-41a0df593ed2","Type":"ContainerStarted","Data":"d82321a34c4c457f0526778f91c84b92fe48f9d42c3a15ccee046a403f5771dc"} Jul 21 13:45:16.990971 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.989471 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xvrvl" event={"ID":"093b3a4b-ec79-4a4d-a86f-05ea62a88e2c","Type":"ContainerStarted","Data":"bb459853b2d697ecd16ee1c4ee58cb613ad707b6ceaef29247af838e170679fa"} Jul 21 13:45:16.993569 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.993363 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 13:45:16.993569 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.993516 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" event={"ID":"cd83129f-d2e9-4efc-99be-4bc5af88e534","Type":"ContainerStarted","Data":"cc93125e8034ba1ef4031a5e4d6c00e4cd40ef0a9411faacfe7021959a570ceb"} Jul 21 13:45:16.994484 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.994461 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:45:16.996266 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.996231 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v7j4w" event={"ID":"c52474d7-8c91-4256-bd47-fd1aef601222","Type":"ContainerStarted","Data":"db3f44ac3a2d42d87b7846db8761cc6e3978295c0fca1194e4ed5d82c4a25e67"} Jul 21 13:45:16.998871 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.998652 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" event={"ID":"9ea20305-07d4-404e-91f7-fe2c5ef68598","Type":"ContainerStarted","Data":"91f32030dd9740f57521367210b24e4236dbc58420ebb691c1637b78321dc701"} Jul 21 13:45:16.998871 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:16.998740 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" event={"ID":"9ea20305-07d4-404e-91f7-fe2c5ef68598","Type":"ContainerStarted","Data":"37f9d09967c1a8a7bb6d5a611970c9786a0d6514262a4c2aebc24cbe589368ed"} Jul 21 13:45:17.000506 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.000482 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-pdjqz" Jul 21 13:45:17.000749 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.000730 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xlpnd" event={"ID":"d08982d1-044f-462c-a84a-d4c65f1cbbef","Type":"ContainerStarted","Data":"f5673230115ac6b0598910b646ddce0080be1591978612e547d7e9f97b0d892c"} Jul 21 13:45:17.000825 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.000767 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xlpnd" event={"ID":"d08982d1-044f-462c-a84a-d4c65f1cbbef","Type":"ContainerStarted","Data":"97ccb1c137476ce871cc00afb0f2583d0325f71fb779b28af401453f587dbbd4"} Jul 21 13:45:17.014398 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.014331 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dtmfw" podStartSLOduration=34.718360951 podStartE2EDuration="43.014308901s" totalImagesPullingTime="8.29594795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:17.013325885 +0000 UTC m=+78.165167004" watchObservedRunningTime="2026-07-21 13:45:17.014308901 +0000 UTC m=+78.166150011" Jul 21 13:45:17.045130 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.045077 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-m57g8" podStartSLOduration=40.808544829 podStartE2EDuration="49.04505934s" totalImagesPullingTime="8.236514511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:17.044440645 +0000 UTC m=+78.196281756" watchObservedRunningTime="2026-07-21 13:45:17.04505934 +0000 UTC m=+78.196900448" Jul 21 13:45:17.070716 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.070652 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:17.099324 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.099263 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-kfq8l" podStartSLOduration=42.214408336 podStartE2EDuration="44.09924151s" totalImagesPullingTime="1.884833174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:17.098085927 +0000 UTC m=+78.249927040" watchObservedRunningTime="2026-07-21 13:45:17.09924151 +0000 UTC m=+78.251082619" Jul 21 13:45:17.126966 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.126898 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ng6jq" podStartSLOduration=40.661232324 podStartE2EDuration="49.126873747s" totalImagesPullingTime="8.465641423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:17.125295828 +0000 UTC m=+78.277136938" watchObservedRunningTime="2026-07-21 13:45:17.126873747 +0000 UTC m=+78.278714865" Jul 21 13:45:17.169069 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.169011 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xlpnd" podStartSLOduration=69.641301126 podStartE2EDuration="1m17.168991123s" totalImagesPullingTime="7.527689997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:17.165477007 +0000 UTC m=+78.317318118" watchObservedRunningTime="2026-07-21 13:45:17.168991123 +0000 UTC m=+78.320832232" Jul 21 13:45:17.218240 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.218140 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:17.224879 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.224818 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xvrvl" podStartSLOduration=68.980365741 podStartE2EDuration="1m17.224797014s" totalImagesPullingTime="8.244431273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:17.223928243 +0000 UTC m=+78.375769352" watchObservedRunningTime="2026-07-21 13:45:17.224797014 +0000 UTC m=+78.376638123" Jul 21 13:45:17.306164 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.306099 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:17.412144 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.412091 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:17.505751 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.505609 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:17.581386 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.581338 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:17.804607 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.804454 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:17.872625 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:17.872588 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:18.203074 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:18.203030 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:18.298445 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:18.298395 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:18.408105 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:18.408068 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:18.507132 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:18.507033 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:18.820761 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:18.820667 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:19.008944 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:19.008873 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:19.197190 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:19.197156 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:19.634680 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:19.634588 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:19.848874 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:19.848838 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:19.949690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:19.949640 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:20.020321 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:20.020287 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:20.662725 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:20.662668 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:21.025432 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:21.025328 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v7j4w" event={"ID":"c52474d7-8c91-4256-bd47-fd1aef601222","Type":"ContainerStarted","Data":"298264024a7dbbed273d151337ab6b8b3f310c06cd14aa5ed2100b30f9830ba1"} Jul 21 13:45:21.045813 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:21.045757 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v7j4w" podStartSLOduration=16.985473922 podStartE2EDuration="35.045740681s" totalImagesPullingTime="18.060266759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:21.045282686 +0000 UTC m=+82.197123797" watchObservedRunningTime="2026-07-21 13:45:21.045740681 +0000 UTC m=+82.197581790" Jul 21 13:45:22.772380 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:22.772252 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:22.833183 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:22.833144 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:22.974530 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:22.974489 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:23.074485 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:23.074393 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:25.192646 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:25.192573 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:25.298293 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:25.298261 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:25.570973 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:25.570870 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:28.017687 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:28.017655 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dtmfw" Jul 21 13:45:28.948839 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:28.948811 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:45:29.108947 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:29.108897 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:29.237602 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:29.237502 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:30.452349 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:30.452306 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:30.552277 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:30.552227 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:32.827954 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:32.827917 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:32.889038 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:32.889005 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:33.811662 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:33.811623 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-789fb75d98-pqr2q"] Jul 21 13:45:33.909513 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:33.909459 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:33.965613 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:33.965570 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:34.021606 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:34.021556 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:34.081058 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:34.080964 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:36.636529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:36.636477 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:36.900348 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:36.900295 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:37.099656 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:37.099569 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:37.198124 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:37.198043 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:37.250329 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:37.250295 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:38.141459 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.141426 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:38.242215 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.242175 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:38.379506 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.379456 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:38.477240 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.477205 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:38.680806 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.680764 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6zpbn"] Jul 21 13:45:38.796171 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.796089 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.798988 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.798965 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 13:45:38.800378 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.800351 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t6whq\"" Jul 21 13:45:38.800378 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.800370 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 13:45:38.800560 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.800392 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 13:45:38.800560 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.800417 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 13:45:38.891544 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.891512 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-root\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.891544 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.891550 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-sys\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.891808 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.891573 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-textfile\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.891808 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.891673 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-accelerators-collector-config\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.891808 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.891766 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-metrics-client-ca\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.891968 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.891817 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.891968 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.891842 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqbt7\" (UniqueName: \"kubernetes.io/projected/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-kube-api-access-pqbt7\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.891968 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.891874 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-tls\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.891968 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.891897 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-wtmp\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.946262 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.946232 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2cps2" Jul 21 13:45:38.992489 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.992442 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.992489 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.992476 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqbt7\" (UniqueName: \"kubernetes.io/projected/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-kube-api-access-pqbt7\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.992770 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.992503 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-tls\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.992770 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.992526 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-wtmp\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.992770 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.992586 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-root\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.992770 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.992608 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-sys\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.992770 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.992631 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-textfile\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.992770 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.992669 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-accelerators-collector-config\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.992770 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.992722 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-metrics-client-ca\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.993132 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.992921 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-root\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.993132 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.993088 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-sys\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.993240 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.993205 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-textfile\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.993354 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.993314 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-wtmp\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.993678 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.993650 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-accelerators-collector-config\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.993880 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.993774 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-metrics-client-ca\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.995754 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.995731 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-tls\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:38.996893 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:38.996873 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:39.003391 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:39.003355 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqbt7\" (UniqueName: \"kubernetes.io/projected/5a9e8cac-01bb-4fd0-9f41-3e8512e23572-kube-api-access-pqbt7\") pod \"node-exporter-6zpbn\" (UID: \"5a9e8cac-01bb-4fd0-9f41-3e8512e23572\") " pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:39.107530 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:39.107447 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6zpbn" Jul 21 13:45:39.118947 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:45:39.118917 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a9e8cac_01bb_4fd0_9f41_3e8512e23572.slice/crio-36c63417be3f607d4f10fab5633613b957a944ce22c4ea2e078ecd769d72ec23 WatchSource:0}: Error finding container 36c63417be3f607d4f10fab5633613b957a944ce22c4ea2e078ecd769d72ec23: Status 404 returned error can't find the container with id 36c63417be3f607d4f10fab5633613b957a944ce22c4ea2e078ecd769d72ec23 Jul 21 13:45:39.671194 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:39.671160 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:39.750982 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:39.750942 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:39.969144 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:39.969040 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:40.110072 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:40.110034 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6zpbn" event={"ID":"5a9e8cac-01bb-4fd0-9f41-3e8512e23572","Type":"ContainerStarted","Data":"36c63417be3f607d4f10fab5633613b957a944ce22c4ea2e078ecd769d72ec23"} Jul 21 13:45:41.115413 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:41.115378 2567 generic.go:356] "Generic (PLEG): container finished" podID="5a9e8cac-01bb-4fd0-9f41-3e8512e23572" containerID="a2221e00946efd5fdb5c494845a989f3b72b3bfa4d9ce70c5537b7e2bff56085" exitCode=0 Jul 21 13:45:41.115907 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:41.115431 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6zpbn" event={"ID":"5a9e8cac-01bb-4fd0-9f41-3e8512e23572","Type":"ContainerDied","Data":"a2221e00946efd5fdb5c494845a989f3b72b3bfa4d9ce70c5537b7e2bff56085"} Jul 21 13:45:41.582436 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:41.582392 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:41.639759 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:41.639721 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:41.730819 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:41.730782 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:41.838907 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:41.838818 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:42.121182 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:42.121088 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6zpbn" event={"ID":"5a9e8cac-01bb-4fd0-9f41-3e8512e23572","Type":"ContainerStarted","Data":"7efe6a746b80dcbd461f55c32a408cba763c54a2241ddf1ed95489f12aacb40b"} Jul 21 13:45:42.121182 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:42.121156 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6zpbn" event={"ID":"5a9e8cac-01bb-4fd0-9f41-3e8512e23572","Type":"ContainerStarted","Data":"182d8c7a0b2047f13d12f6ec3140ab0df1b0ec78b9c2f7307e80d501873c1824"} Jul 21 13:45:42.144308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:42.144245 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6zpbn" podStartSLOduration=2.86592252 podStartE2EDuration="4.144223742s" totalImagesPullingTime="1.211926257s" totalInitContainerRuntime="66.374965ms" isStatefulPod=true podCreationTimestamp="2026-07-21 13:45:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:42.144147807 +0000 UTC m=+103.295988917" watchObservedRunningTime="2026-07-21 13:45:42.144223742 +0000 UTC m=+103.296064851" Jul 21 13:45:42.302123 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:42.302061 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:42.394287 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:42.394195 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:43.365183 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:43.365150 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:43.426535 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:43.426496 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:44.861683 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:44.861646 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:44.927033 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:44.926998 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:46.154874 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:46.154835 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:46.220753 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:46.220711 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:48.954919 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:48.954881 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:49.131933 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:49.131900 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:49.201385 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:49.201345 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:56.175255 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:56.175220 2567 generic.go:356] "Generic (PLEG): container finished" podID="8d80093e-2150-45d5-b78a-a96386daf7e9" containerID="bf4bd18e79d8d2323db048ea9651bb1ece0c153fd6c877f5fadafedfba3d3ed4" exitCode=0 Jul 21 13:45:56.175737 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:56.175298 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-65wbr" event={"ID":"8d80093e-2150-45d5-b78a-a96386daf7e9","Type":"ContainerDied","Data":"bf4bd18e79d8d2323db048ea9651bb1ece0c153fd6c877f5fadafedfba3d3ed4"} Jul 21 13:45:56.175737 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:56.175692 2567 scope.go:117] "RemoveContainer" containerID="bf4bd18e79d8d2323db048ea9651bb1ece0c153fd6c877f5fadafedfba3d3ed4" Jul 21 13:45:56.176800 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:56.176777 2567 generic.go:356] "Generic (PLEG): container finished" podID="80d7f298-9397-475b-951d-baf26856c339" containerID="c22eb3c92d99b62a3a74d2ab0f3daec5ad58bb7a104614c1570fd75c19ce7844" exitCode=0 Jul 21 13:45:56.176882 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:56.176835 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" event={"ID":"80d7f298-9397-475b-951d-baf26856c339","Type":"ContainerDied","Data":"c22eb3c92d99b62a3a74d2ab0f3daec5ad58bb7a104614c1570fd75c19ce7844"} Jul 21 13:45:56.177127 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:56.177112 2567 scope.go:117] "RemoveContainer" containerID="c22eb3c92d99b62a3a74d2ab0f3daec5ad58bb7a104614c1570fd75c19ce7844" Jul 21 13:45:56.453879 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:56.449023 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:56.549552 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:56.549512 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:57.182610 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:57.182567 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-p56gh" event={"ID":"80d7f298-9397-475b-951d-baf26856c339","Type":"ContainerStarted","Data":"c2aad00970e5fa8dd5014f197cb1ba6e02cc472469e3a092c4f305d0f10bb7d9"} Jul 21 13:45:57.184318 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:57.184293 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-65wbr" event={"ID":"8d80093e-2150-45d5-b78a-a96386daf7e9","Type":"ContainerStarted","Data":"eeafb913f6d1722ad43429e3aa290a8da8441f1685fe992067485906248ff5f2"} Jul 21 13:45:57.664753 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:57.664688 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:58.833104 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:58.833060 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" podUID="3b4040af-d0ae-46c2-ab55-880e8f31af17" containerName="registry" containerID="cri-o://82f8b3ab575ca41c2942ecb3167568a7edd118a73896a0ddfbfba8c4ea1e0e51" gracePeriod=30 Jul 21 13:45:58.944348 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:58.944284 2567 patch_prober.go:28] interesting pod/image-registry-789fb75d98-pqr2q container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.132.0.16:5000/healthz\": dial tcp 10.132.0.16:5000: connect: connection refused" start-of-body= Jul 21 13:45:58.944524 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:58.944424 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" podUID="3b4040af-d0ae-46c2-ab55-880e8f31af17" containerName="registry" probeResult="failure" output="Get \"https://10.132.0.16:5000/healthz\": dial tcp 10.132.0.16:5000: connect: connection refused" Jul 21 13:45:59.073664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.073639 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:45:59.152505 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.152476 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3b4040af-d0ae-46c2-ab55-880e8f31af17-image-registry-private-configuration\") pod \"3b4040af-d0ae-46c2-ab55-880e8f31af17\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " Jul 21 13:45:59.152662 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.152513 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-bound-sa-token\") pod \"3b4040af-d0ae-46c2-ab55-880e8f31af17\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " Jul 21 13:45:59.152662 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.152564 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3b4040af-d0ae-46c2-ab55-880e8f31af17-ca-trust-extracted\") pod \"3b4040af-d0ae-46c2-ab55-880e8f31af17\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " Jul 21 13:45:59.152662 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.152587 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3b4040af-d0ae-46c2-ab55-880e8f31af17-trusted-ca\") pod \"3b4040af-d0ae-46c2-ab55-880e8f31af17\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " Jul 21 13:45:59.152662 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.152607 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jchn\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-kube-api-access-2jchn\") pod \"3b4040af-d0ae-46c2-ab55-880e8f31af17\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " Jul 21 13:45:59.152662 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.152640 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3b4040af-d0ae-46c2-ab55-880e8f31af17-installation-pull-secrets\") pod \"3b4040af-d0ae-46c2-ab55-880e8f31af17\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " Jul 21 13:45:59.152662 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.152661 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls\") pod \"3b4040af-d0ae-46c2-ab55-880e8f31af17\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " Jul 21 13:45:59.152984 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.152687 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-certificates\") pod \"3b4040af-d0ae-46c2-ab55-880e8f31af17\" (UID: \"3b4040af-d0ae-46c2-ab55-880e8f31af17\") " Jul 21 13:45:59.153166 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.153140 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b4040af-d0ae-46c2-ab55-880e8f31af17-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3b4040af-d0ae-46c2-ab55-880e8f31af17" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:45:59.153224 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.153192 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3b4040af-d0ae-46c2-ab55-880e8f31af17" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:45:59.155321 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.155268 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b4040af-d0ae-46c2-ab55-880e8f31af17-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3b4040af-d0ae-46c2-ab55-880e8f31af17" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:45:59.155432 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.155326 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b4040af-d0ae-46c2-ab55-880e8f31af17-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3b4040af-d0ae-46c2-ab55-880e8f31af17" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:45:59.155432 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.155325 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3b4040af-d0ae-46c2-ab55-880e8f31af17" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:45:59.155432 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.155404 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-kube-api-access-2jchn" (OuterVolumeSpecName: "kube-api-access-2jchn") pod "3b4040af-d0ae-46c2-ab55-880e8f31af17" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17"). InnerVolumeSpecName "kube-api-access-2jchn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:45:59.155546 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.155519 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3b4040af-d0ae-46c2-ab55-880e8f31af17" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:45:59.160979 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.160952 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b4040af-d0ae-46c2-ab55-880e8f31af17-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3b4040af-d0ae-46c2-ab55-880e8f31af17" (UID: "3b4040af-d0ae-46c2-ab55-880e8f31af17"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:45:59.193248 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.193215 2567 generic.go:356] "Generic (PLEG): container finished" podID="3b4040af-d0ae-46c2-ab55-880e8f31af17" containerID="82f8b3ab575ca41c2942ecb3167568a7edd118a73896a0ddfbfba8c4ea1e0e51" exitCode=0 Jul 21 13:45:59.193381 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.193260 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" Jul 21 13:45:59.193381 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.193302 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" event={"ID":"3b4040af-d0ae-46c2-ab55-880e8f31af17","Type":"ContainerDied","Data":"82f8b3ab575ca41c2942ecb3167568a7edd118a73896a0ddfbfba8c4ea1e0e51"} Jul 21 13:45:59.193381 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.193365 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-789fb75d98-pqr2q" event={"ID":"3b4040af-d0ae-46c2-ab55-880e8f31af17","Type":"ContainerDied","Data":"4e3e8bddce4d4ef5db6957f6dd6a26d95d5e5e02ca78105e7c00578be556541f"} Jul 21 13:45:59.193505 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.193387 2567 scope.go:117] "RemoveContainer" containerID="82f8b3ab575ca41c2942ecb3167568a7edd118a73896a0ddfbfba8c4ea1e0e51" Jul 21 13:45:59.202159 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.202146 2567 scope.go:117] "RemoveContainer" containerID="82f8b3ab575ca41c2942ecb3167568a7edd118a73896a0ddfbfba8c4ea1e0e51" Jul 21 13:45:59.202403 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:45:59.202382 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82f8b3ab575ca41c2942ecb3167568a7edd118a73896a0ddfbfba8c4ea1e0e51\": container with ID starting with 82f8b3ab575ca41c2942ecb3167568a7edd118a73896a0ddfbfba8c4ea1e0e51 not found: ID does not exist" containerID="82f8b3ab575ca41c2942ecb3167568a7edd118a73896a0ddfbfba8c4ea1e0e51" Jul 21 13:45:59.202445 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.202412 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82f8b3ab575ca41c2942ecb3167568a7edd118a73896a0ddfbfba8c4ea1e0e51"} err="failed to get container status \"82f8b3ab575ca41c2942ecb3167568a7edd118a73896a0ddfbfba8c4ea1e0e51\": rpc error: code = NotFound desc = could not find container \"82f8b3ab575ca41c2942ecb3167568a7edd118a73896a0ddfbfba8c4ea1e0e51\": container with ID starting with 82f8b3ab575ca41c2942ecb3167568a7edd118a73896a0ddfbfba8c4ea1e0e51 not found: ID does not exist" Jul 21 13:45:59.214989 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.214969 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-789fb75d98-pqr2q"] Jul 21 13:45:59.218996 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.218975 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-789fb75d98-pqr2q"] Jul 21 13:45:59.253830 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.253795 2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3b4040af-d0ae-46c2-ab55-880e8f31af17-ca-trust-extracted\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:45:59.253830 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.253824 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3b4040af-d0ae-46c2-ab55-880e8f31af17-trusted-ca\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:45:59.253830 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.253835 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jchn\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-kube-api-access-2jchn\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:45:59.254013 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.253843 2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3b4040af-d0ae-46c2-ab55-880e8f31af17-installation-pull-secrets\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:45:59.254013 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.253852 2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-tls\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:45:59.254013 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.253860 2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3b4040af-d0ae-46c2-ab55-880e8f31af17-registry-certificates\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:45:59.254013 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.253869 2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3b4040af-d0ae-46c2-ab55-880e8f31af17-image-registry-private-configuration\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:45:59.254013 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.253878 2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b4040af-d0ae-46c2-ab55-880e8f31af17-bound-sa-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:45:59.531564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.531490 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b4040af-d0ae-46c2-ab55-880e8f31af17" path="/var/lib/kubelet/pods/3b4040af-d0ae-46c2-ab55-880e8f31af17/volumes" Jul 21 13:45:59.844465 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.844378 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:45:59.945343 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:45:59.945307 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:01.161625 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.161584 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bbdc75576-4t7ld"] Jul 21 13:46:01.162046 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.162016 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b4040af-d0ae-46c2-ab55-880e8f31af17" containerName="registry" Jul 21 13:46:01.162046 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.162032 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b4040af-d0ae-46c2-ab55-880e8f31af17" containerName="registry" Jul 21 13:46:01.162150 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.162089 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b4040af-d0ae-46c2-ab55-880e8f31af17" containerName="registry" Jul 21 13:46:01.167449 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.167418 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.175204 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.175178 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 21 13:46:01.175553 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.175531 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 21 13:46:01.176831 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.176807 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 21 13:46:01.180363 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.180342 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 21 13:46:01.180520 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.180504 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 21 13:46:01.180731 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.180715 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 21 13:46:01.180783 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.180738 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-68n6s\"" Jul 21 13:46:01.180838 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.180821 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 21 13:46:01.183372 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.183356 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 21 13:46:01.235810 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.235778 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bbdc75576-4t7ld"] Jul 21 13:46:01.269722 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.269671 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-service-ca\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.269887 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.269730 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-serving-cert\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.269887 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.269758 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-trusted-ca-bundle\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.269887 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.269778 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp6zb\" (UniqueName: \"kubernetes.io/projected/e475cfc6-f958-4683-8c84-6fec8b8f2262-kube-api-access-wp6zb\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.269887 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.269796 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-oauth-config\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.269887 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.269847 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-oauth-serving-cert\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.270085 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.269888 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-config\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.371165 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.371130 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-service-ca\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.371340 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.371174 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-serving-cert\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.371340 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.371192 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-trusted-ca-bundle\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.371340 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.371208 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp6zb\" (UniqueName: \"kubernetes.io/projected/e475cfc6-f958-4683-8c84-6fec8b8f2262-kube-api-access-wp6zb\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.371340 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.371234 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-oauth-config\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.371340 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.371266 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-oauth-serving-cert\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.371340 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.371291 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-config\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.372168 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.372136 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-config\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.372307 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.372177 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-oauth-serving-cert\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.372307 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.372186 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-trusted-ca-bundle\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.372307 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.372234 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-service-ca\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.373855 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.373824 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-oauth-config\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.374425 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.374406 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-serving-cert\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.381363 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.381341 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp6zb\" (UniqueName: \"kubernetes.io/projected/e475cfc6-f958-4683-8c84-6fec8b8f2262-kube-api-access-wp6zb\") pod \"console-5bbdc75576-4t7ld\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.479630 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.479540 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:01.616915 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.616892 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bbdc75576-4t7ld"] Jul 21 13:46:01.619275 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:46:01.619253 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode475cfc6_f958_4683_8c84_6fec8b8f2262.slice/crio-02e1f8f55dd560a82fc5ca986baba25d5f5497b4f9b5d675a99cc7c90fb5fb8f WatchSource:0}: Error finding container 02e1f8f55dd560a82fc5ca986baba25d5f5497b4f9b5d675a99cc7c90fb5fb8f: Status 404 returned error can't find the container with id 02e1f8f55dd560a82fc5ca986baba25d5f5497b4f9b5d675a99cc7c90fb5fb8f Jul 21 13:46:01.695148 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.695110 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:01.776316 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:01.776216 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:02.055044 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:02.054954 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:02.156995 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:02.156954 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:02.205859 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:02.205824 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bbdc75576-4t7ld" event={"ID":"e475cfc6-f958-4683-8c84-6fec8b8f2262","Type":"ContainerStarted","Data":"02e1f8f55dd560a82fc5ca986baba25d5f5497b4f9b5d675a99cc7c90fb5fb8f"} Jul 21 13:46:05.221099 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:05.221059 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bbdc75576-4t7ld" event={"ID":"e475cfc6-f958-4683-8c84-6fec8b8f2262","Type":"ContainerStarted","Data":"91a30c2ab63b2945528d5516f8632fb888a958cf96189e69938dfd88f32bfed5"} Jul 21 13:46:05.241599 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:05.241543 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bbdc75576-4t7ld" podStartSLOduration=1.188807132 podStartE2EDuration="4.241524034s" totalImagesPullingTime="3.052716902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:46:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:46:05.241230032 +0000 UTC m=+126.393071141" watchObservedRunningTime="2026-07-21 13:46:05.241524034 +0000 UTC m=+126.393365137" Jul 21 13:46:11.479956 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:11.479895 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:11.480436 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:11.479980 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:11.485234 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:11.485203 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:12.255530 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:12.255498 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:46:12.410304 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:12.410231 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:12.516266 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:12.516085 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:12.613653 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:12.613612 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:15.024719 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:15.024663 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:15.084850 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:15.084805 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:15.265221 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:15.265185 2567 generic.go:356] "Generic (PLEG): container finished" podID="108a7342-129a-4511-bc08-2b15d4407afa" containerID="0304df1e3d1e9a1056e79f4b3e38d62411a3a2c39eafb01307bd28787cc80c03" exitCode=0 Jul 21 13:46:15.265372 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:15.265251 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" event={"ID":"108a7342-129a-4511-bc08-2b15d4407afa","Type":"ContainerDied","Data":"0304df1e3d1e9a1056e79f4b3e38d62411a3a2c39eafb01307bd28787cc80c03"} Jul 21 13:46:15.265624 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:15.265609 2567 scope.go:117] "RemoveContainer" containerID="0304df1e3d1e9a1056e79f4b3e38d62411a3a2c39eafb01307bd28787cc80c03" Jul 21 13:46:16.271424 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:16.271376 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-zpdl9" event={"ID":"108a7342-129a-4511-bc08-2b15d4407afa","Type":"ContainerStarted","Data":"3847da5a70ce1c48d4cec5715ba25c72e265d34ca575a0c77d7d137bbf03b7c1"} Jul 21 13:46:17.796087 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:17.796040 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:17.861124 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:17.861082 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:39.863366 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:39.863324 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:54.741342 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:54.741247 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:46:54.798806 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:46:54.798768 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:47:02.210303 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:02.210260 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:47:06.792722 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:06.792668 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:47:06.846741 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:06.846693 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:47:14.845637 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:14.845597 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:47:16.500185 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:16.500145 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:47:17.134967 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.134924 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6997c89b48-hcdmm"] Jul 21 13:47:17.138739 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.138692 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.150399 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.150361 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6997c89b48-hcdmm"] Jul 21 13:47:17.191551 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.191501 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-oauth-serving-cert\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.191551 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.191561 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-service-ca\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.191841 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.191582 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-trusted-ca-bundle\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.191841 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.191602 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btmdh\" (UniqueName: \"kubernetes.io/projected/e0d58ce6-e413-4350-8a19-3df19927dda5-kube-api-access-btmdh\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.191841 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.191620 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0d58ce6-e413-4350-8a19-3df19927dda5-console-oauth-config\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.191841 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.191642 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0d58ce6-e413-4350-8a19-3df19927dda5-console-serving-cert\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.191841 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.191664 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-console-config\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.222293 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.222255 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:47:17.288041 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.288004 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:47:17.293063 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.293028 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0d58ce6-e413-4350-8a19-3df19927dda5-console-oauth-config\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.293204 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.293079 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0d58ce6-e413-4350-8a19-3df19927dda5-console-serving-cert\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.293204 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.293106 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-console-config\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.293204 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.293143 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-oauth-serving-cert\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.293204 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.293183 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-service-ca\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.293404 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.293211 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-trusted-ca-bundle\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.293404 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.293306 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btmdh\" (UniqueName: \"kubernetes.io/projected/e0d58ce6-e413-4350-8a19-3df19927dda5-kube-api-access-btmdh\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.294101 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.294075 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-service-ca\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.294233 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.294123 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-oauth-serving-cert\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.294297 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.294274 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-console-config\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.294297 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.294286 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-trusted-ca-bundle\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.295859 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.295829 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0d58ce6-e413-4350-8a19-3df19927dda5-console-oauth-config\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.295985 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.295967 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0d58ce6-e413-4350-8a19-3df19927dda5-console-serving-cert\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.302476 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.302447 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btmdh\" (UniqueName: \"kubernetes.io/projected/e0d58ce6-e413-4350-8a19-3df19927dda5-kube-api-access-btmdh\") pod \"console-6997c89b48-hcdmm\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.451391 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.451349 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:17.594971 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:17.594931 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6997c89b48-hcdmm"] Jul 21 13:47:17.599872 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:47:17.599832 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d58ce6_e413_4350_8a19_3df19927dda5.slice/crio-f522933aa6edf1eb9f66040381823647baaa44d4e0cdcdc3bda129cf964d287b WatchSource:0}: Error finding container f522933aa6edf1eb9f66040381823647baaa44d4e0cdcdc3bda129cf964d287b: Status 404 returned error can't find the container with id f522933aa6edf1eb9f66040381823647baaa44d4e0cdcdc3bda129cf964d287b Jul 21 13:47:18.507465 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:18.507424 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6997c89b48-hcdmm" event={"ID":"e0d58ce6-e413-4350-8a19-3df19927dda5","Type":"ContainerStarted","Data":"ca2c95a08e77284063b6e59437a599e26b18b1e68ad391d578ec2ab348ec2677"} Jul 21 13:47:18.507626 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:18.507495 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6997c89b48-hcdmm" event={"ID":"e0d58ce6-e413-4350-8a19-3df19927dda5","Type":"ContainerStarted","Data":"f522933aa6edf1eb9f66040381823647baaa44d4e0cdcdc3bda129cf964d287b"} Jul 21 13:47:18.525416 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:18.525356 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6997c89b48-hcdmm" podStartSLOduration=1.525337887 podStartE2EDuration="1.525337887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:47:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:47:18.524604968 +0000 UTC m=+199.676446078" watchObservedRunningTime="2026-07-21 13:47:18.525337887 +0000 UTC m=+199.677178995" Jul 21 13:47:27.251839 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:27.251806 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:47:27.311341 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:27.311295 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:47:27.451946 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:27.451901 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:27.452150 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:27.452011 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:27.456904 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:27.456874 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:27.548520 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:27.548441 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:47:27.598961 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:27.598925 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bbdc75576-4t7ld"] Jul 21 13:47:27.710875 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:27.710835 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:47:27.772227 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:27.772185 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:47:37.358878 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:37.358829 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:47:52.622600 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:52.622532 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5bbdc75576-4t7ld" podUID="e475cfc6-f958-4683-8c84-6fec8b8f2262" containerName="console" containerID="cri-o://91a30c2ab63b2945528d5516f8632fb888a958cf96189e69938dfd88f32bfed5" gracePeriod=15 Jul 21 13:47:52.874836 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:52.874759 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:47:53.002482 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.002437 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-config\") pod \"e475cfc6-f958-4683-8c84-6fec8b8f2262\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " Jul 21 13:47:53.002667 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.002506 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-service-ca\") pod \"e475cfc6-f958-4683-8c84-6fec8b8f2262\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " Jul 21 13:47:53.002667 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.002539 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-trusted-ca-bundle\") pod \"e475cfc6-f958-4683-8c84-6fec8b8f2262\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " Jul 21 13:47:53.002667 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.002578 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-oauth-config\") pod \"e475cfc6-f958-4683-8c84-6fec8b8f2262\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " Jul 21 13:47:53.002667 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.002617 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-serving-cert\") pod \"e475cfc6-f958-4683-8c84-6fec8b8f2262\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " Jul 21 13:47:53.002667 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.002659 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-oauth-serving-cert\") pod \"e475cfc6-f958-4683-8c84-6fec8b8f2262\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " Jul 21 13:47:53.002898 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.002747 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp6zb\" (UniqueName: \"kubernetes.io/projected/e475cfc6-f958-4683-8c84-6fec8b8f2262-kube-api-access-wp6zb\") pod \"e475cfc6-f958-4683-8c84-6fec8b8f2262\" (UID: \"e475cfc6-f958-4683-8c84-6fec8b8f2262\") " Jul 21 13:47:53.003006 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.002983 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-config" (OuterVolumeSpecName: "console-config") pod "e475cfc6-f958-4683-8c84-6fec8b8f2262" (UID: "e475cfc6-f958-4683-8c84-6fec8b8f2262"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:47:53.003177 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.003133 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-service-ca" (OuterVolumeSpecName: "service-ca") pod "e475cfc6-f958-4683-8c84-6fec8b8f2262" (UID: "e475cfc6-f958-4683-8c84-6fec8b8f2262"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:47:53.003177 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.003162 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e475cfc6-f958-4683-8c84-6fec8b8f2262" (UID: "e475cfc6-f958-4683-8c84-6fec8b8f2262"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:47:53.003371 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.003243 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e475cfc6-f958-4683-8c84-6fec8b8f2262" (UID: "e475cfc6-f958-4683-8c84-6fec8b8f2262"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:47:53.005202 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.005156 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e475cfc6-f958-4683-8c84-6fec8b8f2262" (UID: "e475cfc6-f958-4683-8c84-6fec8b8f2262"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:47:53.005202 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.005192 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e475cfc6-f958-4683-8c84-6fec8b8f2262-kube-api-access-wp6zb" (OuterVolumeSpecName: "kube-api-access-wp6zb") pod "e475cfc6-f958-4683-8c84-6fec8b8f2262" (UID: "e475cfc6-f958-4683-8c84-6fec8b8f2262"). InnerVolumeSpecName "kube-api-access-wp6zb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:47:53.005357 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.005165 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e475cfc6-f958-4683-8c84-6fec8b8f2262" (UID: "e475cfc6-f958-4683-8c84-6fec8b8f2262"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:47:53.104149 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.104105 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-oauth-config\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:47:53.104149 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.104141 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-serving-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:47:53.104149 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.104151 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-oauth-serving-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:47:53.104149 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.104160 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp6zb\" (UniqueName: \"kubernetes.io/projected/e475cfc6-f958-4683-8c84-6fec8b8f2262-kube-api-access-wp6zb\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:47:53.104428 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.104170 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-console-config\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:47:53.104428 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.104180 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-service-ca\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:47:53.104428 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.104187 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e475cfc6-f958-4683-8c84-6fec8b8f2262-trusted-ca-bundle\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:47:53.646302 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.646264 2567 generic.go:356] "Generic (PLEG): container finished" podID="e475cfc6-f958-4683-8c84-6fec8b8f2262" containerID="91a30c2ab63b2945528d5516f8632fb888a958cf96189e69938dfd88f32bfed5" exitCode=0 Jul 21 13:47:53.646786 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.646315 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bbdc75576-4t7ld" Jul 21 13:47:53.646786 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.646328 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bbdc75576-4t7ld" event={"ID":"e475cfc6-f958-4683-8c84-6fec8b8f2262","Type":"ContainerDied","Data":"91a30c2ab63b2945528d5516f8632fb888a958cf96189e69938dfd88f32bfed5"} Jul 21 13:47:53.646786 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.646383 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bbdc75576-4t7ld" event={"ID":"e475cfc6-f958-4683-8c84-6fec8b8f2262","Type":"ContainerDied","Data":"02e1f8f55dd560a82fc5ca986baba25d5f5497b4f9b5d675a99cc7c90fb5fb8f"} Jul 21 13:47:53.646786 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.646404 2567 scope.go:117] "RemoveContainer" containerID="91a30c2ab63b2945528d5516f8632fb888a958cf96189e69938dfd88f32bfed5" Jul 21 13:47:53.655500 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.655480 2567 scope.go:117] "RemoveContainer" containerID="91a30c2ab63b2945528d5516f8632fb888a958cf96189e69938dfd88f32bfed5" Jul 21 13:47:53.655920 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:47:53.655897 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91a30c2ab63b2945528d5516f8632fb888a958cf96189e69938dfd88f32bfed5\": container with ID starting with 91a30c2ab63b2945528d5516f8632fb888a958cf96189e69938dfd88f32bfed5 not found: ID does not exist" containerID="91a30c2ab63b2945528d5516f8632fb888a958cf96189e69938dfd88f32bfed5" Jul 21 13:47:53.655994 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.655934 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91a30c2ab63b2945528d5516f8632fb888a958cf96189e69938dfd88f32bfed5"} err="failed to get container status \"91a30c2ab63b2945528d5516f8632fb888a958cf96189e69938dfd88f32bfed5\": rpc error: code = NotFound desc = could not find container \"91a30c2ab63b2945528d5516f8632fb888a958cf96189e69938dfd88f32bfed5\": container with ID starting with 91a30c2ab63b2945528d5516f8632fb888a958cf96189e69938dfd88f32bfed5 not found: ID does not exist" Jul 21 13:47:53.668363 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.668307 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bbdc75576-4t7ld"] Jul 21 13:47:53.673643 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:53.673608 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bbdc75576-4t7ld"] Jul 21 13:47:55.533179 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:47:55.532322 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e475cfc6-f958-4683-8c84-6fec8b8f2262" path="/var/lib/kubelet/pods/e475cfc6-f958-4683-8c84-6fec8b8f2262/volumes" Jul 21 13:48:13.792128 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:13.792061 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:13.886585 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:13.886550 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:14.581583 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.581540 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf"] Jul 21 13:48:14.581890 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.581877 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e475cfc6-f958-4683-8c84-6fec8b8f2262" containerName="console" Jul 21 13:48:14.581890 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.581891 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e475cfc6-f958-4683-8c84-6fec8b8f2262" containerName="console" Jul 21 13:48:14.581974 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.581958 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e475cfc6-f958-4683-8c84-6fec8b8f2262" containerName="console" Jul 21 13:48:14.586519 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.586491 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" Jul 21 13:48:14.589688 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.589665 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-95frf\"" Jul 21 13:48:14.590061 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.590038 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 21 13:48:14.590970 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.590953 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 21 13:48:14.594127 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.594094 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf"] Jul 21 13:48:14.777841 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.777796 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee195b16-4a0b-4f56-835d-deb096313fb2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf\" (UID: \"ee195b16-4a0b-4f56-835d-deb096313fb2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" Jul 21 13:48:14.778027 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.777888 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8f64\" (UniqueName: \"kubernetes.io/projected/ee195b16-4a0b-4f56-835d-deb096313fb2-kube-api-access-h8f64\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf\" (UID: \"ee195b16-4a0b-4f56-835d-deb096313fb2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" Jul 21 13:48:14.778027 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.777928 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee195b16-4a0b-4f56-835d-deb096313fb2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf\" (UID: \"ee195b16-4a0b-4f56-835d-deb096313fb2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" Jul 21 13:48:14.878851 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.878763 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8f64\" (UniqueName: \"kubernetes.io/projected/ee195b16-4a0b-4f56-835d-deb096313fb2-kube-api-access-h8f64\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf\" (UID: \"ee195b16-4a0b-4f56-835d-deb096313fb2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" Jul 21 13:48:14.878851 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.878821 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee195b16-4a0b-4f56-835d-deb096313fb2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf\" (UID: \"ee195b16-4a0b-4f56-835d-deb096313fb2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" Jul 21 13:48:14.879233 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.878855 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee195b16-4a0b-4f56-835d-deb096313fb2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf\" (UID: \"ee195b16-4a0b-4f56-835d-deb096313fb2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" Jul 21 13:48:14.879233 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.879178 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee195b16-4a0b-4f56-835d-deb096313fb2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf\" (UID: \"ee195b16-4a0b-4f56-835d-deb096313fb2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" Jul 21 13:48:14.879317 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.879262 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee195b16-4a0b-4f56-835d-deb096313fb2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf\" (UID: \"ee195b16-4a0b-4f56-835d-deb096313fb2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" Jul 21 13:48:14.888055 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.888021 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8f64\" (UniqueName: \"kubernetes.io/projected/ee195b16-4a0b-4f56-835d-deb096313fb2-kube-api-access-h8f64\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf\" (UID: \"ee195b16-4a0b-4f56-835d-deb096313fb2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" Jul 21 13:48:14.898832 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:14.898802 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" Jul 21 13:48:15.039827 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:15.039789 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf"] Jul 21 13:48:15.043159 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:48:15.043131 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee195b16_4a0b_4f56_835d_deb096313fb2.slice/crio-fb8cac6320466735664b94f0998c8e8e37ab8d503afb56c7dc61323301349b62 WatchSource:0}: Error finding container fb8cac6320466735664b94f0998c8e8e37ab8d503afb56c7dc61323301349b62: Status 404 returned error can't find the container with id fb8cac6320466735664b94f0998c8e8e37ab8d503afb56c7dc61323301349b62 Jul 21 13:48:15.734143 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:15.734102 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" event={"ID":"ee195b16-4a0b-4f56-835d-deb096313fb2","Type":"ContainerStarted","Data":"fb8cac6320466735664b94f0998c8e8e37ab8d503afb56c7dc61323301349b62"} Jul 21 13:48:21.760216 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:21.760166 2567 generic.go:356] "Generic (PLEG): container finished" podID="ee195b16-4a0b-4f56-835d-deb096313fb2" containerID="375fc23a0868888e0bfe25a6a59432bcab179a7c8c1cc988f67f387b9722cebc" exitCode=0 Jul 21 13:48:21.760603 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:21.760251 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" event={"ID":"ee195b16-4a0b-4f56-835d-deb096313fb2","Type":"ContainerDied","Data":"375fc23a0868888e0bfe25a6a59432bcab179a7c8c1cc988f67f387b9722cebc"} Jul 21 13:48:24.774013 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:24.773980 2567 generic.go:356] "Generic (PLEG): container finished" podID="ee195b16-4a0b-4f56-835d-deb096313fb2" containerID="72fa7ecd52b932694a727b4dc7c6abbf3304a54a83d77ffaff7a5bd03ab45db0" exitCode=0 Jul 21 13:48:24.774432 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:24.774025 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" event={"ID":"ee195b16-4a0b-4f56-835d-deb096313fb2","Type":"ContainerDied","Data":"72fa7ecd52b932694a727b4dc7c6abbf3304a54a83d77ffaff7a5bd03ab45db0"} Jul 21 13:48:32.809315 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:32.809276 2567 generic.go:356] "Generic (PLEG): container finished" podID="ee195b16-4a0b-4f56-835d-deb096313fb2" containerID="f565e491b8915dea1b8cdbe25f303524aee6c02530e1c0c91486b5c8eaa82aff" exitCode=0 Jul 21 13:48:32.809689 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:32.809332 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" event={"ID":"ee195b16-4a0b-4f56-835d-deb096313fb2","Type":"ContainerDied","Data":"f565e491b8915dea1b8cdbe25f303524aee6c02530e1c0c91486b5c8eaa82aff"} Jul 21 13:48:33.956034 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:33.956003 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" Jul 21 13:48:34.034480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:34.034422 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee195b16-4a0b-4f56-835d-deb096313fb2-bundle\") pod \"ee195b16-4a0b-4f56-835d-deb096313fb2\" (UID: \"ee195b16-4a0b-4f56-835d-deb096313fb2\") " Jul 21 13:48:34.034686 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:34.034569 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8f64\" (UniqueName: \"kubernetes.io/projected/ee195b16-4a0b-4f56-835d-deb096313fb2-kube-api-access-h8f64\") pod \"ee195b16-4a0b-4f56-835d-deb096313fb2\" (UID: \"ee195b16-4a0b-4f56-835d-deb096313fb2\") " Jul 21 13:48:34.034686 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:34.034597 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee195b16-4a0b-4f56-835d-deb096313fb2-util\") pod \"ee195b16-4a0b-4f56-835d-deb096313fb2\" (UID: \"ee195b16-4a0b-4f56-835d-deb096313fb2\") " Jul 21 13:48:34.035186 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:34.035154 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee195b16-4a0b-4f56-835d-deb096313fb2-bundle" (OuterVolumeSpecName: "bundle") pod "ee195b16-4a0b-4f56-835d-deb096313fb2" (UID: "ee195b16-4a0b-4f56-835d-deb096313fb2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:48:34.036868 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:34.036828 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee195b16-4a0b-4f56-835d-deb096313fb2-kube-api-access-h8f64" (OuterVolumeSpecName: "kube-api-access-h8f64") pod "ee195b16-4a0b-4f56-835d-deb096313fb2" (UID: "ee195b16-4a0b-4f56-835d-deb096313fb2"). InnerVolumeSpecName "kube-api-access-h8f64". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:48:34.039348 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:34.039316 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee195b16-4a0b-4f56-835d-deb096313fb2-util" (OuterVolumeSpecName: "util") pod "ee195b16-4a0b-4f56-835d-deb096313fb2" (UID: "ee195b16-4a0b-4f56-835d-deb096313fb2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:48:34.135744 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:34.135612 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee195b16-4a0b-4f56-835d-deb096313fb2-bundle\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:48:34.135744 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:34.135662 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8f64\" (UniqueName: \"kubernetes.io/projected/ee195b16-4a0b-4f56-835d-deb096313fb2-kube-api-access-h8f64\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:48:34.135744 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:34.135678 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee195b16-4a0b-4f56-835d-deb096313fb2-util\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:48:34.818795 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:34.818765 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" Jul 21 13:48:34.818968 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:34.818759 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lcbqf" event={"ID":"ee195b16-4a0b-4f56-835d-deb096313fb2","Type":"ContainerDied","Data":"fb8cac6320466735664b94f0998c8e8e37ab8d503afb56c7dc61323301349b62"} Jul 21 13:48:34.818968 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:34.818893 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb8cac6320466735664b94f0998c8e8e37ab8d503afb56c7dc61323301349b62" Jul 21 13:48:40.651243 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:40.651203 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:41.639975 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:41.639939 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:42.595021 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.594985 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4"] Jul 21 13:48:42.595411 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.595258 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee195b16-4a0b-4f56-835d-deb096313fb2" containerName="util" Jul 21 13:48:42.595411 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.595269 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee195b16-4a0b-4f56-835d-deb096313fb2" containerName="util" Jul 21 13:48:42.595411 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.595293 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee195b16-4a0b-4f56-835d-deb096313fb2" containerName="pull" Jul 21 13:48:42.595411 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.595298 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee195b16-4a0b-4f56-835d-deb096313fb2" containerName="pull" Jul 21 13:48:42.595411 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.595305 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee195b16-4a0b-4f56-835d-deb096313fb2" containerName="extract" Jul 21 13:48:42.595411 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.595320 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee195b16-4a0b-4f56-835d-deb096313fb2" containerName="extract" Jul 21 13:48:42.595411 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.595380 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee195b16-4a0b-4f56-835d-deb096313fb2" containerName="extract" Jul 21 13:48:42.600215 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.600196 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4" Jul 21 13:48:42.603160 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.603133 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7sgcq\"" Jul 21 13:48:42.603160 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.603146 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 21 13:48:42.603160 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.603156 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 21 13:48:42.614161 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.614133 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4"] Jul 21 13:48:42.704898 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.704861 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0db196fe-b951-41b0-9147-d0333ca55de2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-h5gm4\" (UID: \"0db196fe-b951-41b0-9147-d0333ca55de2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4" Jul 21 13:48:42.704898 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.704901 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72vvk\" (UniqueName: \"kubernetes.io/projected/0db196fe-b951-41b0-9147-d0333ca55de2-kube-api-access-72vvk\") pod \"cert-manager-operator-controller-manager-55c444bf86-h5gm4\" (UID: \"0db196fe-b951-41b0-9147-d0333ca55de2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4" Jul 21 13:48:42.805469 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.805426 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0db196fe-b951-41b0-9147-d0333ca55de2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-h5gm4\" (UID: \"0db196fe-b951-41b0-9147-d0333ca55de2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4" Jul 21 13:48:42.805613 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.805480 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72vvk\" (UniqueName: \"kubernetes.io/projected/0db196fe-b951-41b0-9147-d0333ca55de2-kube-api-access-72vvk\") pod \"cert-manager-operator-controller-manager-55c444bf86-h5gm4\" (UID: \"0db196fe-b951-41b0-9147-d0333ca55de2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4" Jul 21 13:48:42.805842 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.805825 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0db196fe-b951-41b0-9147-d0333ca55de2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-h5gm4\" (UID: \"0db196fe-b951-41b0-9147-d0333ca55de2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4" Jul 21 13:48:42.814414 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.814384 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72vvk\" (UniqueName: \"kubernetes.io/projected/0db196fe-b951-41b0-9147-d0333ca55de2-kube-api-access-72vvk\") pod \"cert-manager-operator-controller-manager-55c444bf86-h5gm4\" (UID: \"0db196fe-b951-41b0-9147-d0333ca55de2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4" Jul 21 13:48:42.910067 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:42.910038 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4" Jul 21 13:48:43.049155 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:43.049118 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4"] Jul 21 13:48:43.053153 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:48:43.053125 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db196fe_b951_41b0_9147_d0333ca55de2.slice/crio-61a5268830eb22e2fef5e61d83b6dd6acbdbc1bdab8403febd79d40edf5cfb1c WatchSource:0}: Error finding container 61a5268830eb22e2fef5e61d83b6dd6acbdbc1bdab8403febd79d40edf5cfb1c: Status 404 returned error can't find the container with id 61a5268830eb22e2fef5e61d83b6dd6acbdbc1bdab8403febd79d40edf5cfb1c Jul 21 13:48:43.855866 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:43.855821 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4" event={"ID":"0db196fe-b951-41b0-9147-d0333ca55de2","Type":"ContainerStarted","Data":"61a5268830eb22e2fef5e61d83b6dd6acbdbc1bdab8403febd79d40edf5cfb1c"} Jul 21 13:48:44.264575 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:44.264540 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:44.365188 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:44.365137 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:44.867748 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:44.867680 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4" event={"ID":"0db196fe-b951-41b0-9147-d0333ca55de2","Type":"ContainerStarted","Data":"be04bd4ef4923af492c1bfc812352fbdd241b57e23fa3fc653ed954002aa1c3b"} Jul 21 13:48:44.893915 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:44.893854 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-h5gm4" podStartSLOduration=1.239405533 podStartE2EDuration="2.893836249s" totalImagesPullingTime="1.654430716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:48:44.89276226 +0000 UTC m=+286.044603369" watchObservedRunningTime="2026-07-21 13:48:44.893836249 +0000 UTC m=+286.045677357" Jul 21 13:48:45.302899 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:45.302861 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:45.481075 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:45.481034 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:46.313709 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.313659 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl"] Jul 21 13:48:46.317325 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.317296 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" Jul 21 13:48:46.322061 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.322024 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 21 13:48:46.322233 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.322105 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-95frf\"" Jul 21 13:48:46.323047 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.323025 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 21 13:48:46.337034 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.337000 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl"] Jul 21 13:48:46.436319 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.436280 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl\" (UID: \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" Jul 21 13:48:46.436537 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.436340 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl\" (UID: \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" Jul 21 13:48:46.436537 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.436373 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzfwq\" (UniqueName: \"kubernetes.io/projected/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-kube-api-access-qzfwq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl\" (UID: \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" Jul 21 13:48:46.501962 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.501920 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:46.537306 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.537262 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl\" (UID: \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" Jul 21 13:48:46.537521 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.537318 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzfwq\" (UniqueName: \"kubernetes.io/projected/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-kube-api-access-qzfwq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl\" (UID: \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" Jul 21 13:48:46.537521 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.537368 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl\" (UID: \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" Jul 21 13:48:46.537806 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.537782 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl\" (UID: \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" Jul 21 13:48:46.537881 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.537849 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl\" (UID: \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" Jul 21 13:48:46.551297 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.551259 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzfwq\" (UniqueName: \"kubernetes.io/projected/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-kube-api-access-qzfwq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl\" (UID: \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" Jul 21 13:48:46.596341 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.596233 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:46.627202 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.627163 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" Jul 21 13:48:46.751378 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.751343 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nxcdr"] Jul 21 13:48:46.755849 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.755817 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nxcdr" Jul 21 13:48:46.758793 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.758759 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-tbvl5\"" Jul 21 13:48:46.758978 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.758959 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 21 13:48:46.759330 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.759294 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 21 13:48:46.766538 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.766507 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nxcdr"] Jul 21 13:48:46.773615 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.773586 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl"] Jul 21 13:48:46.776189 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:48:46.776160 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf50a4eb_7707_4fe7_ac98_d651ecbf3d0b.slice/crio-96635b6e475082bf6d75b834a40e61cda1daf7e05957e31744a82a2f18fc3949 WatchSource:0}: Error finding container 96635b6e475082bf6d75b834a40e61cda1daf7e05957e31744a82a2f18fc3949: Status 404 returned error can't find the container with id 96635b6e475082bf6d75b834a40e61cda1daf7e05957e31744a82a2f18fc3949 Jul 21 13:48:46.840759 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.840724 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd75e5c2-cbd4-41a8-9ed7-9ced283a724c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nxcdr\" (UID: \"bd75e5c2-cbd4-41a8-9ed7-9ced283a724c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nxcdr" Jul 21 13:48:46.840915 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.840797 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpcng\" (UniqueName: \"kubernetes.io/projected/bd75e5c2-cbd4-41a8-9ed7-9ced283a724c-kube-api-access-jpcng\") pod \"cert-manager-webhook-6486ffd997-nxcdr\" (UID: \"bd75e5c2-cbd4-41a8-9ed7-9ced283a724c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nxcdr" Jul 21 13:48:46.879082 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.879047 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" event={"ID":"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b","Type":"ContainerStarted","Data":"2072f79284c828fc4db05548035de6468b4822cb18665e3f1d479e7068814164"} Jul 21 13:48:46.879253 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.879097 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" event={"ID":"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b","Type":"ContainerStarted","Data":"96635b6e475082bf6d75b834a40e61cda1daf7e05957e31744a82a2f18fc3949"} Jul 21 13:48:46.941667 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.941624 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpcng\" (UniqueName: \"kubernetes.io/projected/bd75e5c2-cbd4-41a8-9ed7-9ced283a724c-kube-api-access-jpcng\") pod \"cert-manager-webhook-6486ffd997-nxcdr\" (UID: \"bd75e5c2-cbd4-41a8-9ed7-9ced283a724c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nxcdr" Jul 21 13:48:46.941875 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.941756 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd75e5c2-cbd4-41a8-9ed7-9ced283a724c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nxcdr\" (UID: \"bd75e5c2-cbd4-41a8-9ed7-9ced283a724c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nxcdr" Jul 21 13:48:46.950515 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.950482 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd75e5c2-cbd4-41a8-9ed7-9ced283a724c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nxcdr\" (UID: \"bd75e5c2-cbd4-41a8-9ed7-9ced283a724c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nxcdr" Jul 21 13:48:46.950666 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:46.950547 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpcng\" (UniqueName: \"kubernetes.io/projected/bd75e5c2-cbd4-41a8-9ed7-9ced283a724c-kube-api-access-jpcng\") pod \"cert-manager-webhook-6486ffd997-nxcdr\" (UID: \"bd75e5c2-cbd4-41a8-9ed7-9ced283a724c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nxcdr" Jul 21 13:48:47.075488 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:47.075450 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nxcdr" Jul 21 13:48:47.214020 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:47.213990 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nxcdr"] Jul 21 13:48:47.216388 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:48:47.216362 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd75e5c2_cbd4_41a8_9ed7_9ced283a724c.slice/crio-2cc67da9ebb26f8aed8d5b2e20e5ab3b53c5ad0e3c7a8691e8126e631788a3df WatchSource:0}: Error finding container 2cc67da9ebb26f8aed8d5b2e20e5ab3b53c5ad0e3c7a8691e8126e631788a3df: Status 404 returned error can't find the container with id 2cc67da9ebb26f8aed8d5b2e20e5ab3b53c5ad0e3c7a8691e8126e631788a3df Jul 21 13:48:47.885026 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:47.884935 2567 generic.go:356] "Generic (PLEG): container finished" podID="bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b" containerID="2072f79284c828fc4db05548035de6468b4822cb18665e3f1d479e7068814164" exitCode=0 Jul 21 13:48:47.885026 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:47.885005 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" event={"ID":"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b","Type":"ContainerDied","Data":"2072f79284c828fc4db05548035de6468b4822cb18665e3f1d479e7068814164"} Jul 21 13:48:47.886587 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:47.886477 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nxcdr" event={"ID":"bd75e5c2-cbd4-41a8-9ed7-9ced283a724c","Type":"ContainerStarted","Data":"2cc67da9ebb26f8aed8d5b2e20e5ab3b53c5ad0e3c7a8691e8126e631788a3df"} Jul 21 13:48:48.649066 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:48.649029 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:49.058426 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:49.058315 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:50.905308 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:50.905268 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" event={"ID":"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b","Type":"ContainerStarted","Data":"9e4266fbc1d28b638bcfcaf234feaa1a5327d115a25b29406f328b60ae9debb1"} Jul 21 13:48:50.907023 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:50.906987 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nxcdr" event={"ID":"bd75e5c2-cbd4-41a8-9ed7-9ced283a724c","Type":"ContainerStarted","Data":"c1a222548d11d6bb9ff5e5bdc6a02269d41850cb8b12902afa02e7bc1ca939f0"} Jul 21 13:48:50.907169 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:50.907137 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nxcdr" Jul 21 13:48:50.954401 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:50.954357 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-nxcdr" podStartSLOduration=1.4002709420000001 podStartE2EDuration="4.954340904s" totalImagesPullingTime="3.554069962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:48:50.953852421 +0000 UTC m=+292.105693530" watchObservedRunningTime="2026-07-21 13:48:50.954340904 +0000 UTC m=+292.106182013" Jul 21 13:48:51.052956 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:51.052868 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:51.153466 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:51.153424 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:51.912935 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:51.912900 2567 generic.go:356] "Generic (PLEG): container finished" podID="bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b" containerID="9e4266fbc1d28b638bcfcaf234feaa1a5327d115a25b29406f328b60ae9debb1" exitCode=0 Jul 21 13:48:51.913309 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:51.912987 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" event={"ID":"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b","Type":"ContainerDied","Data":"9e4266fbc1d28b638bcfcaf234feaa1a5327d115a25b29406f328b60ae9debb1"} Jul 21 13:48:52.920850 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:52.920815 2567 generic.go:356] "Generic (PLEG): container finished" podID="bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b" containerID="5dd9bd0c1ec072352cf6e201aa4c498433ead31db51bc84f676cc8c74551b476" exitCode=0 Jul 21 13:48:52.921228 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:52.920901 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" event={"ID":"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b","Type":"ContainerDied","Data":"5dd9bd0c1ec072352cf6e201aa4c498433ead31db51bc84f676cc8c74551b476"} Jul 21 13:48:54.059408 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.059371 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" Jul 21 13:48:54.201456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.201358 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-util\") pod \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\" (UID: \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\") " Jul 21 13:48:54.201456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.201430 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-bundle\") pod \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\" (UID: \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\") " Jul 21 13:48:54.201686 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.201490 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzfwq\" (UniqueName: \"kubernetes.io/projected/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-kube-api-access-qzfwq\") pod \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\" (UID: \"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b\") " Jul 21 13:48:54.201869 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.201840 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-bundle" (OuterVolumeSpecName: "bundle") pod "bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b" (UID: "bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:48:54.203746 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.203717 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-kube-api-access-qzfwq" (OuterVolumeSpecName: "kube-api-access-qzfwq") pod "bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b" (UID: "bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b"). InnerVolumeSpecName "kube-api-access-qzfwq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:48:54.207197 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.207152 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-util" (OuterVolumeSpecName: "util") pod "bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b" (UID: "bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:48:54.282572 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.282530 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:54.302583 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.302544 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzfwq\" (UniqueName: \"kubernetes.io/projected/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-kube-api-access-qzfwq\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:48:54.302583 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.302579 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-util\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:48:54.302583 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.302592 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b-bundle\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:48:54.380858 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.380818 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:54.932527 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.932494 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" Jul 21 13:48:54.932856 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.932492 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7h7fl" event={"ID":"bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b","Type":"ContainerDied","Data":"96635b6e475082bf6d75b834a40e61cda1daf7e05957e31744a82a2f18fc3949"} Jul 21 13:48:54.932856 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:54.932649 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96635b6e475082bf6d75b834a40e61cda1daf7e05957e31744a82a2f18fc3949" Jul 21 13:48:56.662487 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:56.662446 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:56.915319 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:56.915223 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nxcdr" Jul 21 13:48:58.998867 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:58.998812 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:59.097506 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:59.097470 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:48:59.285605 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:59.285515 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 13:48:59.289737 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:59.289690 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 13:48:59.305199 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:48:59.305172 2567 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 13:49:00.316456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.316415 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4"] Jul 21 13:49:00.329166 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.316753 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b" containerName="pull" Jul 21 13:49:00.329166 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.316766 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b" containerName="pull" Jul 21 13:49:00.329166 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.316774 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b" containerName="extract" Jul 21 13:49:00.329166 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.316780 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b" containerName="extract" Jul 21 13:49:00.329166 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.316790 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b" containerName="util" Jul 21 13:49:00.329166 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.316796 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b" containerName="util" Jul 21 13:49:00.329166 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.316870 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf50a4eb-7707-4fe7-ac98-d651ecbf3d0b" containerName="extract" Jul 21 13:49:00.335441 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.335371 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4"] Jul 21 13:49:00.335602 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.335466 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4" Jul 21 13:49:00.338445 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.338422 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 21 13:49:00.338591 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.338425 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 21 13:49:00.339686 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.339654 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-xztmh\"" Jul 21 13:49:00.455017 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.454972 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aff57af5-3ad0-419d-a2d9-c8cc29a7e5db-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tqdg4\" (UID: \"aff57af5-3ad0-419d-a2d9-c8cc29a7e5db\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4" Jul 21 13:49:00.455202 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.455064 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8bcw\" (UniqueName: \"kubernetes.io/projected/aff57af5-3ad0-419d-a2d9-c8cc29a7e5db-kube-api-access-n8bcw\") pod \"openshift-lws-operator-fd8ccff4c-tqdg4\" (UID: \"aff57af5-3ad0-419d-a2d9-c8cc29a7e5db\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4" Jul 21 13:49:00.556273 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.556227 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8bcw\" (UniqueName: \"kubernetes.io/projected/aff57af5-3ad0-419d-a2d9-c8cc29a7e5db-kube-api-access-n8bcw\") pod \"openshift-lws-operator-fd8ccff4c-tqdg4\" (UID: \"aff57af5-3ad0-419d-a2d9-c8cc29a7e5db\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4" Jul 21 13:49:00.556456 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.556318 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aff57af5-3ad0-419d-a2d9-c8cc29a7e5db-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tqdg4\" (UID: \"aff57af5-3ad0-419d-a2d9-c8cc29a7e5db\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4" Jul 21 13:49:00.556782 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.556751 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aff57af5-3ad0-419d-a2d9-c8cc29a7e5db-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tqdg4\" (UID: \"aff57af5-3ad0-419d-a2d9-c8cc29a7e5db\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4" Jul 21 13:49:00.566122 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.566080 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8bcw\" (UniqueName: \"kubernetes.io/projected/aff57af5-3ad0-419d-a2d9-c8cc29a7e5db-kube-api-access-n8bcw\") pod \"openshift-lws-operator-fd8ccff4c-tqdg4\" (UID: \"aff57af5-3ad0-419d-a2d9-c8cc29a7e5db\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4" Jul 21 13:49:00.645162 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.645066 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4" Jul 21 13:49:00.814527 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.814498 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4"] Jul 21 13:49:00.817143 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:49:00.817103 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaff57af5_3ad0_419d_a2d9_c8cc29a7e5db.slice/crio-03ade6327dafe89fc1c4fac81e2adb8c5ab95dcc5c3866f614e9b63c00d41308 WatchSource:0}: Error finding container 03ade6327dafe89fc1c4fac81e2adb8c5ab95dcc5c3866f614e9b63c00d41308: Status 404 returned error can't find the container with id 03ade6327dafe89fc1c4fac81e2adb8c5ab95dcc5c3866f614e9b63c00d41308 Jul 21 13:49:00.962944 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:00.962908 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4" event={"ID":"aff57af5-3ad0-419d-a2d9-c8cc29a7e5db","Type":"ContainerStarted","Data":"03ade6327dafe89fc1c4fac81e2adb8c5ab95dcc5c3866f614e9b63c00d41308"} Jul 21 13:49:03.979058 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:03.979001 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4" event={"ID":"aff57af5-3ad0-419d-a2d9-c8cc29a7e5db","Type":"ContainerStarted","Data":"4434292e76390bd7e7e9869fc9c88845d53656214eda8eb6317ebf3b2c3a2a68"} Jul 21 13:49:03.999331 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:03.999245 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tqdg4" podStartSLOduration=1.356945372 podStartE2EDuration="3.999223927s" totalImagesPullingTime="2.642278555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:49:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:49:03.996602864 +0000 UTC m=+305.148443973" watchObservedRunningTime="2026-07-21 13:49:03.999223927 +0000 UTC m=+305.151065035" Jul 21 13:49:04.926833 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:04.926788 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:06.889203 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:06.889163 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9"] Jul 21 13:49:06.891983 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:06.891960 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" Jul 21 13:49:06.896689 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:06.896661 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 21 13:49:06.896880 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:06.896691 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-95frf\"" Jul 21 13:49:06.898482 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:06.898449 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 21 13:49:06.904707 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:06.904662 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9"] Jul 21 13:49:07.015866 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.015814 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lpjn\" (UniqueName: \"kubernetes.io/projected/f311fa9c-233b-4eab-bfdb-95b2e65e7275-kube-api-access-8lpjn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9\" (UID: \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" Jul 21 13:49:07.015866 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.015861 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f311fa9c-233b-4eab-bfdb-95b2e65e7275-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9\" (UID: \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" Jul 21 13:49:07.016132 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.015893 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f311fa9c-233b-4eab-bfdb-95b2e65e7275-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9\" (UID: \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" Jul 21 13:49:07.117083 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.117042 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lpjn\" (UniqueName: \"kubernetes.io/projected/f311fa9c-233b-4eab-bfdb-95b2e65e7275-kube-api-access-8lpjn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9\" (UID: \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" Jul 21 13:49:07.117083 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.117081 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f311fa9c-233b-4eab-bfdb-95b2e65e7275-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9\" (UID: \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" Jul 21 13:49:07.117313 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.117105 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f311fa9c-233b-4eab-bfdb-95b2e65e7275-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9\" (UID: \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" Jul 21 13:49:07.117495 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.117474 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f311fa9c-233b-4eab-bfdb-95b2e65e7275-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9\" (UID: \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" Jul 21 13:49:07.117532 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.117481 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f311fa9c-233b-4eab-bfdb-95b2e65e7275-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9\" (UID: \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" Jul 21 13:49:07.129889 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.129856 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lpjn\" (UniqueName: \"kubernetes.io/projected/f311fa9c-233b-4eab-bfdb-95b2e65e7275-kube-api-access-8lpjn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9\" (UID: \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" Jul 21 13:49:07.202895 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.202848 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" Jul 21 13:49:07.354615 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.354586 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9"] Jul 21 13:49:07.357268 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:49:07.357236 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf311fa9c_233b_4eab_bfdb_95b2e65e7275.slice/crio-ad3d226a98568951bda2fc48f3354aa06b812b37381d054c1723918545e9611b WatchSource:0}: Error finding container ad3d226a98568951bda2fc48f3354aa06b812b37381d054c1723918545e9611b: Status 404 returned error can't find the container with id ad3d226a98568951bda2fc48f3354aa06b812b37381d054c1723918545e9611b Jul 21 13:49:07.676142 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.676106 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:07.998607 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.998567 2567 generic.go:356] "Generic (PLEG): container finished" podID="f311fa9c-233b-4eab-bfdb-95b2e65e7275" containerID="4fde7df1a45bf931301f3112ea9ebadebcd539b5af2306a29b0456420ba9928d" exitCode=0 Jul 21 13:49:07.999060 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.998661 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" event={"ID":"f311fa9c-233b-4eab-bfdb-95b2e65e7275","Type":"ContainerDied","Data":"4fde7df1a45bf931301f3112ea9ebadebcd539b5af2306a29b0456420ba9928d"} Jul 21 13:49:07.999060 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.998762 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" event={"ID":"f311fa9c-233b-4eab-bfdb-95b2e65e7275","Type":"ContainerStarted","Data":"ad3d226a98568951bda2fc48f3354aa06b812b37381d054c1723918545e9611b"} Jul 21 13:49:07.999157 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:07.999141 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 13:49:11.018270 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:11.018228 2567 generic.go:356] "Generic (PLEG): container finished" podID="f311fa9c-233b-4eab-bfdb-95b2e65e7275" containerID="e7113c446bae5e35ac68346d7c17d3b92af2e2332cb64f7eac320b773b39594a" exitCode=0 Jul 21 13:49:11.018739 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:11.018311 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" event={"ID":"f311fa9c-233b-4eab-bfdb-95b2e65e7275","Type":"ContainerDied","Data":"e7113c446bae5e35ac68346d7c17d3b92af2e2332cb64f7eac320b773b39594a"} Jul 21 13:49:12.025655 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:12.025611 2567 generic.go:356] "Generic (PLEG): container finished" podID="f311fa9c-233b-4eab-bfdb-95b2e65e7275" containerID="f1d1b78b057973252702d182d58690402bfd96e10b397a0fdc1f255fc353cf45" exitCode=0 Jul 21 13:49:12.026084 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:12.025693 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" event={"ID":"f311fa9c-233b-4eab-bfdb-95b2e65e7275","Type":"ContainerDied","Data":"f1d1b78b057973252702d182d58690402bfd96e10b397a0fdc1f255fc353cf45"} Jul 21 13:49:13.174235 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:13.174197 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" Jul 21 13:49:13.272201 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:13.272159 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lpjn\" (UniqueName: \"kubernetes.io/projected/f311fa9c-233b-4eab-bfdb-95b2e65e7275-kube-api-access-8lpjn\") pod \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\" (UID: \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\") " Jul 21 13:49:13.272409 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:13.272232 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f311fa9c-233b-4eab-bfdb-95b2e65e7275-bundle\") pod \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\" (UID: \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\") " Jul 21 13:49:13.272409 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:13.272254 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f311fa9c-233b-4eab-bfdb-95b2e65e7275-util\") pod \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\" (UID: \"f311fa9c-233b-4eab-bfdb-95b2e65e7275\") " Jul 21 13:49:13.273017 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:13.272982 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f311fa9c-233b-4eab-bfdb-95b2e65e7275-bundle" (OuterVolumeSpecName: "bundle") pod "f311fa9c-233b-4eab-bfdb-95b2e65e7275" (UID: "f311fa9c-233b-4eab-bfdb-95b2e65e7275"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:49:13.274431 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:13.274404 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f311fa9c-233b-4eab-bfdb-95b2e65e7275-kube-api-access-8lpjn" (OuterVolumeSpecName: "kube-api-access-8lpjn") pod "f311fa9c-233b-4eab-bfdb-95b2e65e7275" (UID: "f311fa9c-233b-4eab-bfdb-95b2e65e7275"). InnerVolumeSpecName "kube-api-access-8lpjn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:49:13.277203 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:13.277170 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f311fa9c-233b-4eab-bfdb-95b2e65e7275-util" (OuterVolumeSpecName: "util") pod "f311fa9c-233b-4eab-bfdb-95b2e65e7275" (UID: "f311fa9c-233b-4eab-bfdb-95b2e65e7275"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:49:13.373349 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:13.373240 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lpjn\" (UniqueName: \"kubernetes.io/projected/f311fa9c-233b-4eab-bfdb-95b2e65e7275-kube-api-access-8lpjn\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:49:13.373349 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:13.373277 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f311fa9c-233b-4eab-bfdb-95b2e65e7275-bundle\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:49:13.373349 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:13.373287 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f311fa9c-233b-4eab-bfdb-95b2e65e7275-util\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:49:14.037512 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:14.037486 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" Jul 21 13:49:14.037691 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:14.037511 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqf2m9" event={"ID":"f311fa9c-233b-4eab-bfdb-95b2e65e7275","Type":"ContainerDied","Data":"ad3d226a98568951bda2fc48f3354aa06b812b37381d054c1723918545e9611b"} Jul 21 13:49:14.037691 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:14.037555 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad3d226a98568951bda2fc48f3354aa06b812b37381d054c1723918545e9611b" Jul 21 13:49:19.066592 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.066552 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:19.152458 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.152407 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5"] Jul 21 13:49:19.152775 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.152761 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f311fa9c-233b-4eab-bfdb-95b2e65e7275" containerName="pull" Jul 21 13:49:19.152816 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.152777 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f311fa9c-233b-4eab-bfdb-95b2e65e7275" containerName="pull" Jul 21 13:49:19.152816 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.152792 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f311fa9c-233b-4eab-bfdb-95b2e65e7275" containerName="extract" Jul 21 13:49:19.152816 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.152798 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f311fa9c-233b-4eab-bfdb-95b2e65e7275" containerName="extract" Jul 21 13:49:19.152929 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.152816 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f311fa9c-233b-4eab-bfdb-95b2e65e7275" containerName="util" Jul 21 13:49:19.152929 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.152825 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f311fa9c-233b-4eab-bfdb-95b2e65e7275" containerName="util" Jul 21 13:49:19.152929 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.152890 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f311fa9c-233b-4eab-bfdb-95b2e65e7275" containerName="extract" Jul 21 13:49:19.156419 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.156385 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.161379 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.161346 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 21 13:49:19.161541 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.161346 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 21 13:49:19.161541 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.161459 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 21 13:49:19.161541 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.161354 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-h5js4\"" Jul 21 13:49:19.173109 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.173070 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5"] Jul 21 13:49:19.178067 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.178019 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:19.219952 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.219908 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n99k\" (UniqueName: \"kubernetes.io/projected/991d90c7-3003-4ffe-b861-5a7ad42dbf5d-kube-api-access-4n99k\") pod \"lws-controller-manager-686f4b4fc5-mbrz5\" (UID: \"991d90c7-3003-4ffe-b861-5a7ad42dbf5d\") " pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.220215 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.219965 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/991d90c7-3003-4ffe-b861-5a7ad42dbf5d-manager-config\") pod \"lws-controller-manager-686f4b4fc5-mbrz5\" (UID: \"991d90c7-3003-4ffe-b861-5a7ad42dbf5d\") " pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.220215 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.220060 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/991d90c7-3003-4ffe-b861-5a7ad42dbf5d-cert\") pod \"lws-controller-manager-686f4b4fc5-mbrz5\" (UID: \"991d90c7-3003-4ffe-b861-5a7ad42dbf5d\") " pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.220215 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.220115 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/991d90c7-3003-4ffe-b861-5a7ad42dbf5d-metrics-cert\") pod \"lws-controller-manager-686f4b4fc5-mbrz5\" (UID: \"991d90c7-3003-4ffe-b861-5a7ad42dbf5d\") " pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.320935 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.320849 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/991d90c7-3003-4ffe-b861-5a7ad42dbf5d-cert\") pod \"lws-controller-manager-686f4b4fc5-mbrz5\" (UID: \"991d90c7-3003-4ffe-b861-5a7ad42dbf5d\") " pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.320935 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.320892 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/991d90c7-3003-4ffe-b861-5a7ad42dbf5d-metrics-cert\") pod \"lws-controller-manager-686f4b4fc5-mbrz5\" (UID: \"991d90c7-3003-4ffe-b861-5a7ad42dbf5d\") " pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.321114 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.320936 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n99k\" (UniqueName: \"kubernetes.io/projected/991d90c7-3003-4ffe-b861-5a7ad42dbf5d-kube-api-access-4n99k\") pod \"lws-controller-manager-686f4b4fc5-mbrz5\" (UID: \"991d90c7-3003-4ffe-b861-5a7ad42dbf5d\") " pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.321114 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.320967 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/991d90c7-3003-4ffe-b861-5a7ad42dbf5d-manager-config\") pod \"lws-controller-manager-686f4b4fc5-mbrz5\" (UID: \"991d90c7-3003-4ffe-b861-5a7ad42dbf5d\") " pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.321755 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.321727 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/991d90c7-3003-4ffe-b861-5a7ad42dbf5d-manager-config\") pod \"lws-controller-manager-686f4b4fc5-mbrz5\" (UID: \"991d90c7-3003-4ffe-b861-5a7ad42dbf5d\") " pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.323544 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.323518 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/991d90c7-3003-4ffe-b861-5a7ad42dbf5d-metrics-cert\") pod \"lws-controller-manager-686f4b4fc5-mbrz5\" (UID: \"991d90c7-3003-4ffe-b861-5a7ad42dbf5d\") " pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.323659 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.323600 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/991d90c7-3003-4ffe-b861-5a7ad42dbf5d-cert\") pod \"lws-controller-manager-686f4b4fc5-mbrz5\" (UID: \"991d90c7-3003-4ffe-b861-5a7ad42dbf5d\") " pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.330850 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.330816 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n99k\" (UniqueName: \"kubernetes.io/projected/991d90c7-3003-4ffe-b861-5a7ad42dbf5d-kube-api-access-4n99k\") pod \"lws-controller-manager-686f4b4fc5-mbrz5\" (UID: \"991d90c7-3003-4ffe-b861-5a7ad42dbf5d\") " pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.467106 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.467072 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:19.632111 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:19.631892 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5"] Jul 21 13:49:19.634733 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:49:19.634682 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991d90c7_3003_4ffe_b861_5a7ad42dbf5d.slice/crio-0f9963a8d910131e1731b48c0c502eacfd25ea9c5de518aaaa90eba1a09c9580 WatchSource:0}: Error finding container 0f9963a8d910131e1731b48c0c502eacfd25ea9c5de518aaaa90eba1a09c9580: Status 404 returned error can't find the container with id 0f9963a8d910131e1731b48c0c502eacfd25ea9c5de518aaaa90eba1a09c9580 Jul 21 13:49:20.070560 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:20.070512 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" event={"ID":"991d90c7-3003-4ffe-b861-5a7ad42dbf5d","Type":"ContainerStarted","Data":"0f9963a8d910131e1731b48c0c502eacfd25ea9c5de518aaaa90eba1a09c9580"} Jul 21 13:49:22.082776 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:22.082739 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" event={"ID":"991d90c7-3003-4ffe-b861-5a7ad42dbf5d","Type":"ContainerStarted","Data":"fb84b9265997f52ffc19a4a73f330d22dae7ffee8e5aca98773e2468ae847bfb"} Jul 21 13:49:22.083166 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:22.083047 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:22.102090 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:22.102020 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" podStartSLOduration=1.365786678 podStartE2EDuration="3.101994774s" totalImagesPullingTime="1.736208096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:49:22.101141129 +0000 UTC m=+323.252982241" watchObservedRunningTime="2026-07-21 13:49:22.101994774 +0000 UTC m=+323.253835884" Jul 21 13:49:24.850318 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:24.850277 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:24.948211 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:24.948171 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:25.407334 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.407274 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:25.654168 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.654120 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:25.756916 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.756809 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:25.814416 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.814373 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7"] Jul 21 13:49:25.817458 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.817435 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.820520 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.820488 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 13:49:25.820788 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.820605 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 13:49:25.820788 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.820617 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hwcdx\"" Jul 21 13:49:25.820971 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.820815 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 13:49:25.821243 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.821220 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 13:49:25.832270 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.832233 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7"] Jul 21 13:49:25.875752 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.875679 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-manifests\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.876147 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.875765 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.876147 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.875806 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjwlf\" (UniqueName: \"kubernetes.io/projected/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-kube-api-access-mjwlf\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.876147 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.875833 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.876147 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.875860 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-tmp\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.923200 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.923152 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:25.976680 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.976637 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-manifests\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.976680 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.976683 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.976959 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.976744 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjwlf\" (UniqueName: \"kubernetes.io/projected/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-kube-api-access-mjwlf\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.976959 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.976775 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.976959 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.976798 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-tmp\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.977149 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.977124 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-manifests\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.979217 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.979191 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-tmp\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.979562 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.979541 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.979605 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.979541 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:25.985872 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:25.985840 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjwlf\" (UniqueName: \"kubernetes.io/projected/150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6-kube-api-access-mjwlf\") pod \"opendatahub-operator-controller-manager-5d98d8798d-jmhl7\" (UID: \"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:26.016871 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:26.016774 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:26.116818 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:26.116780 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:26.129782 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:26.129741 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:26.219208 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:26.219177 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:26.286476 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:26.286444 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7"] Jul 21 13:49:26.289083 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:49:26.289053 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod150a2a3c_6487_41eb_8c84_7cc1bd8c4dd6.slice/crio-c4393b243f90ea9eac59a75c095439b9941e842ba2ee03267b4a5186e0fe75a3 WatchSource:0}: Error finding container c4393b243f90ea9eac59a75c095439b9941e842ba2ee03267b4a5186e0fe75a3: Status 404 returned error can't find the container with id c4393b243f90ea9eac59a75c095439b9941e842ba2ee03267b4a5186e0fe75a3 Jul 21 13:49:26.357565 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:26.357487 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:27.110437 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:27.110400 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" event={"ID":"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6","Type":"ContainerStarted","Data":"c4393b243f90ea9eac59a75c095439b9941e842ba2ee03267b4a5186e0fe75a3"} Jul 21 13:49:30.135693 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:30.135642 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" event={"ID":"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6","Type":"ContainerStarted","Data":"4a4d86f6064e434d7678179d773f0d80ef3e76134ff7324ae026c033ac0d0f07"} Jul 21 13:49:31.142388 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:31.142293 2567 generic.go:356] "Generic (PLEG): container finished" podID="150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6" containerID="4a4d86f6064e434d7678179d773f0d80ef3e76134ff7324ae026c033ac0d0f07" exitCode=0 Jul 21 13:49:31.142778 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:31.142391 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" event={"ID":"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6","Type":"ContainerDied","Data":"4a4d86f6064e434d7678179d773f0d80ef3e76134ff7324ae026c033ac0d0f07"} Jul 21 13:49:32.149206 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:32.149168 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" event={"ID":"150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6","Type":"ContainerStarted","Data":"eeaa68f6ab6b29b55d34f40a3898281694c51c760423a2f53f536702a7378028"} Jul 21 13:49:32.149621 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:32.149287 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:32.201333 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:32.201271 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" podStartSLOduration=2.277308579 podStartE2EDuration="7.20125196s" totalImagesPullingTime="3.818480027s" totalInitContainerRuntime="1.105463354s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:49:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:49:32.20022523 +0000 UTC m=+333.352066338" watchObservedRunningTime="2026-07-21 13:49:32.20125196 +0000 UTC m=+333.353093068" Jul 21 13:49:33.089644 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:33.089602 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-686f4b4fc5-mbrz5" Jul 21 13:49:41.124517 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:41.124471 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:41.221229 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:41.221195 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:43.157293 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:43.157261 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-jmhl7" Jul 21 13:49:45.480885 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:45.480838 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:45.596131 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:45.596087 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:46.192405 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:46.192360 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:46.273142 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:46.273095 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:46.371619 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:46.371575 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:46.524009 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:46.523910 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:46.845232 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:46.845121 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:47.004931 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:47.004898 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:55.838025 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:55.837777 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:55.940171 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:55.940120 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:56.018616 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:56.018571 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:56.173358 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:56.173322 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:56.442805 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:56.442616 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:56.942949 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:56.942912 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:57.183999 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:57.183959 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:57.344863 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:57.344765 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:58.446804 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:58.446743 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:58.542042 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:58.541997 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:49:59.789601 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:49:59.789564 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:03.759652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:03.759615 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:03.859242 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:03.859200 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:04.440372 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.440333 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx"] Jul 21 13:50:04.450409 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.450375 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.453671 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.453641 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-bjv9x\"" Jul 21 13:50:04.453843 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.453673 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 21 13:50:04.458289 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.458251 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx"] Jul 21 13:50:04.487680 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.487644 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:04.532450 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.532411 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6vgm\" (UniqueName: \"kubernetes.io/projected/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-kube-api-access-m6vgm\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.532622 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.532467 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.532622 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.532501 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.532765 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.532629 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.532765 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.532658 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.532765 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.532712 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.532765 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.532742 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.532896 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.532794 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.532896 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.532863 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.633774 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.633738 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.633994 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.633788 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.633994 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.633828 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.633994 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.633861 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.633994 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.633894 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.633994 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.633934 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.633994 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.633964 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6vgm\" (UniqueName: \"kubernetes.io/projected/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-kube-api-access-m6vgm\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.634309 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.633997 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.634309 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.634029 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.634419 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.634325 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.634419 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.634352 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.634419 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.634406 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.634628 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.634599 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.634628 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.634624 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.636532 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.636511 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.636792 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.636774 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.644352 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.644301 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6vgm\" (UniqueName: \"kubernetes.io/projected/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-kube-api-access-m6vgm\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.644487 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.644387 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-554689689d996fx\" (UID: \"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.686790 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.686748 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:04.764380 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.764260 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:04.923956 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.923898 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx"] Jul 21 13:50:04.927801 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:50:04.927768 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c857ac6_0465_49f4_ae6f_29f5e7cfb2d5.slice/crio-bfa44665c48dc7ede48325bb77d5d55f64e5f91a9f9b8ae268babd8a84362143 WatchSource:0}: Error finding container bfa44665c48dc7ede48325bb77d5d55f64e5f91a9f9b8ae268babd8a84362143: Status 404 returned error can't find the container with id bfa44665c48dc7ede48325bb77d5d55f64e5f91a9f9b8ae268babd8a84362143 Jul 21 13:50:04.962256 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:04.962204 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:05.322918 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:05.322878 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" event={"ID":"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5","Type":"ContainerStarted","Data":"bfa44665c48dc7ede48325bb77d5d55f64e5f91a9f9b8ae268babd8a84362143"} Jul 21 13:50:06.840864 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:06.840796 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:07.059095 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:07.059053 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:07.633325 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:07.633275 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:08.355113 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:08.355070 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:08.518196 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:08.518159 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 13:50:08.518279 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:08.518236 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 13:50:08.518279 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:08.518267 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 13:50:09.352827 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:09.352790 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" event={"ID":"6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5","Type":"ContainerStarted","Data":"fd6d6f0aceb19123be0dc0ec2976f27d9729f756497410ba2c1d3c29e90aa25e"} Jul 21 13:50:09.377005 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:09.376942 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" podStartSLOduration=1.78912457 podStartE2EDuration="5.376924427s" totalImagesPullingTime="3.587799857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:50:09.374333333 +0000 UTC m=+370.526174442" watchObservedRunningTime="2026-07-21 13:50:09.376924427 +0000 UTC m=+370.528765535" Jul 21 13:50:09.764615 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:09.764575 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:09.769728 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:09.769679 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:10.357403 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:10.357367 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:10.358454 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:10.358433 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-554689689d996fx" Jul 21 13:50:29.847078 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:29.847028 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:29.945198 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:29.945156 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:46.924235 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:46.924195 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:47.019266 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.019210 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:47.156162 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.156119 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g6nh2"] Jul 21 13:50:47.170116 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.170084 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-g6nh2" Jul 21 13:50:47.173605 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.173574 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 13:50:47.174729 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.174628 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-fhqp9\"" Jul 21 13:50:47.175067 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.174659 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 13:50:47.175940 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.175911 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g6nh2"] Jul 21 13:50:47.325246 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.325200 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lp8j\" (UniqueName: \"kubernetes.io/projected/bc0c7629-d395-45bf-b7fa-dd963df04047-kube-api-access-9lp8j\") pod \"kuadrant-operator-catalog-g6nh2\" (UID: \"bc0c7629-d395-45bf-b7fa-dd963df04047\") " pod="kuadrant-system/kuadrant-operator-catalog-g6nh2" Jul 21 13:50:47.328372 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.328335 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:47.417841 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.417797 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g6nh2"] Jul 21 13:50:47.418106 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:50:47.418086 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-9lp8j], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-catalog-g6nh2" podUID="bc0c7629-d395-45bf-b7fa-dd963df04047" Jul 21 13:50:47.425778 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.425680 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lp8j\" (UniqueName: \"kubernetes.io/projected/bc0c7629-d395-45bf-b7fa-dd963df04047-kube-api-access-9lp8j\") pod \"kuadrant-operator-catalog-g6nh2\" (UID: \"bc0c7629-d395-45bf-b7fa-dd963df04047\") " pod="kuadrant-system/kuadrant-operator-catalog-g6nh2" Jul 21 13:50:47.438980 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.438946 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lp8j\" (UniqueName: \"kubernetes.io/projected/bc0c7629-d395-45bf-b7fa-dd963df04047-kube-api-access-9lp8j\") pod \"kuadrant-operator-catalog-g6nh2\" (UID: \"bc0c7629-d395-45bf-b7fa-dd963df04047\") " pod="kuadrant-system/kuadrant-operator-catalog-g6nh2" Jul 21 13:50:47.534600 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.534562 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-g6nh2" Jul 21 13:50:47.542622 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.542594 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-g6nh2" Jul 21 13:50:47.621045 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.621008 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4ktmm"] Jul 21 13:50:47.624772 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.624746 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4ktmm" Jul 21 13:50:47.634059 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.634027 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4ktmm"] Jul 21 13:50:47.728103 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.728003 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lp8j\" (UniqueName: \"kubernetes.io/projected/bc0c7629-d395-45bf-b7fa-dd963df04047-kube-api-access-9lp8j\") pod \"bc0c7629-d395-45bf-b7fa-dd963df04047\" (UID: \"bc0c7629-d395-45bf-b7fa-dd963df04047\") " Jul 21 13:50:47.728290 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.728194 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckg4h\" (UniqueName: \"kubernetes.io/projected/e8edc027-509d-4241-91c1-0f3caf4f5a67-kube-api-access-ckg4h\") pod \"kuadrant-operator-catalog-4ktmm\" (UID: \"e8edc027-509d-4241-91c1-0f3caf4f5a67\") " pod="kuadrant-system/kuadrant-operator-catalog-4ktmm" Jul 21 13:50:47.730397 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.730364 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc0c7629-d395-45bf-b7fa-dd963df04047-kube-api-access-9lp8j" (OuterVolumeSpecName: "kube-api-access-9lp8j") pod "bc0c7629-d395-45bf-b7fa-dd963df04047" (UID: "bc0c7629-d395-45bf-b7fa-dd963df04047"). InnerVolumeSpecName "kube-api-access-9lp8j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:50:47.829377 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.829332 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckg4h\" (UniqueName: \"kubernetes.io/projected/e8edc027-509d-4241-91c1-0f3caf4f5a67-kube-api-access-ckg4h\") pod \"kuadrant-operator-catalog-4ktmm\" (UID: \"e8edc027-509d-4241-91c1-0f3caf4f5a67\") " pod="kuadrant-system/kuadrant-operator-catalog-4ktmm" Jul 21 13:50:47.829570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.829501 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lp8j\" (UniqueName: \"kubernetes.io/projected/bc0c7629-d395-45bf-b7fa-dd963df04047-kube-api-access-9lp8j\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:50:47.840273 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.840240 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckg4h\" (UniqueName: \"kubernetes.io/projected/e8edc027-509d-4241-91c1-0f3caf4f5a67-kube-api-access-ckg4h\") pod \"kuadrant-operator-catalog-4ktmm\" (UID: \"e8edc027-509d-4241-91c1-0f3caf4f5a67\") " pod="kuadrant-system/kuadrant-operator-catalog-4ktmm" Jul 21 13:50:47.860296 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.860254 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:47.936276 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:47.936233 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4ktmm" Jul 21 13:50:48.057726 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:48.057664 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:50:48.097601 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:48.097575 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4ktmm"] Jul 21 13:50:48.100038 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:50:48.100011 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8edc027_509d_4241_91c1_0f3caf4f5a67.slice/crio-7f33bc9281d149435ecf1a816cb10275e14be121a4582447cfc79a09b2d10852 WatchSource:0}: Error finding container 7f33bc9281d149435ecf1a816cb10275e14be121a4582447cfc79a09b2d10852: Status 404 returned error can't find the container with id 7f33bc9281d149435ecf1a816cb10275e14be121a4582447cfc79a09b2d10852 Jul 21 13:50:48.541411 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:48.541371 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4ktmm" event={"ID":"e8edc027-509d-4241-91c1-0f3caf4f5a67","Type":"ContainerStarted","Data":"7f33bc9281d149435ecf1a816cb10275e14be121a4582447cfc79a09b2d10852"} Jul 21 13:50:48.541411 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:48.541411 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-g6nh2" Jul 21 13:50:48.578487 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:48.578446 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g6nh2"] Jul 21 13:50:48.582552 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:48.582520 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-g6nh2"] Jul 21 13:50:49.535895 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:49.535843 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc0c7629-d395-45bf-b7fa-dd963df04047" path="/var/lib/kubelet/pods/bc0c7629-d395-45bf-b7fa-dd963df04047/volumes" Jul 21 13:50:50.555376 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:50.555325 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4ktmm" event={"ID":"e8edc027-509d-4241-91c1-0f3caf4f5a67","Type":"ContainerStarted","Data":"e73e328a9fa7d5cf24563e738932480bf95e7cbb779b49d46bc9c3927821f188"} Jul 21 13:50:50.591111 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:50.590992 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4ktmm" podStartSLOduration=1.368964797 podStartE2EDuration="3.590970479s" totalImagesPullingTime="2.222005682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:50:50.588015834 +0000 UTC m=+411.739856939" watchObservedRunningTime="2026-07-21 13:50:50.590970479 +0000 UTC m=+411.742811588" Jul 21 13:50:57.936982 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:57.936932 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-4ktmm" Jul 21 13:50:57.937426 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:57.937013 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-4ktmm" Jul 21 13:50:57.960606 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:57.960572 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-4ktmm" Jul 21 13:50:58.616880 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:50:58.616847 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-4ktmm" Jul 21 13:51:03.046316 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.046278 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv"] Jul 21 13:51:03.050139 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.050109 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" Jul 21 13:51:03.053225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.053192 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-sq6kq\"" Jul 21 13:51:03.060533 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.060499 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv"] Jul 21 13:51:03.150255 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.150217 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/103cc13a-a074-47bf-aeac-68deb81f3520-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv\" (UID: \"103cc13a-a074-47bf-aeac-68deb81f3520\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" Jul 21 13:51:03.150479 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.150279 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/103cc13a-a074-47bf-aeac-68deb81f3520-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv\" (UID: \"103cc13a-a074-47bf-aeac-68deb81f3520\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" Jul 21 13:51:03.150479 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.150336 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dq6q\" (UniqueName: \"kubernetes.io/projected/103cc13a-a074-47bf-aeac-68deb81f3520-kube-api-access-9dq6q\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv\" (UID: \"103cc13a-a074-47bf-aeac-68deb81f3520\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" Jul 21 13:51:03.251490 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.251446 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/103cc13a-a074-47bf-aeac-68deb81f3520-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv\" (UID: \"103cc13a-a074-47bf-aeac-68deb81f3520\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" Jul 21 13:51:03.251685 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.251539 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/103cc13a-a074-47bf-aeac-68deb81f3520-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv\" (UID: \"103cc13a-a074-47bf-aeac-68deb81f3520\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" Jul 21 13:51:03.251685 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.251566 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dq6q\" (UniqueName: \"kubernetes.io/projected/103cc13a-a074-47bf-aeac-68deb81f3520-kube-api-access-9dq6q\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv\" (UID: \"103cc13a-a074-47bf-aeac-68deb81f3520\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" Jul 21 13:51:03.251968 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.251943 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/103cc13a-a074-47bf-aeac-68deb81f3520-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv\" (UID: \"103cc13a-a074-47bf-aeac-68deb81f3520\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" Jul 21 13:51:03.252017 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.251951 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/103cc13a-a074-47bf-aeac-68deb81f3520-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv\" (UID: \"103cc13a-a074-47bf-aeac-68deb81f3520\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" Jul 21 13:51:03.261997 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.261968 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dq6q\" (UniqueName: \"kubernetes.io/projected/103cc13a-a074-47bf-aeac-68deb81f3520-kube-api-access-9dq6q\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv\" (UID: \"103cc13a-a074-47bf-aeac-68deb81f3520\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" Jul 21 13:51:03.362036 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.361936 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" Jul 21 13:51:03.517264 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.517227 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv"] Jul 21 13:51:03.517403 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:51:03.517235 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103cc13a_a074_47bf_aeac_68deb81f3520.slice/crio-4fca1af6f7c56c98c79a6526c7cdacee4ebf47d7b6475a6326e375fd120b9f66 WatchSource:0}: Error finding container 4fca1af6f7c56c98c79a6526c7cdacee4ebf47d7b6475a6326e375fd120b9f66: Status 404 returned error can't find the container with id 4fca1af6f7c56c98c79a6526c7cdacee4ebf47d7b6475a6326e375fd120b9f66 Jul 21 13:51:03.620209 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.620171 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" event={"ID":"103cc13a-a074-47bf-aeac-68deb81f3520","Type":"ContainerStarted","Data":"13aa66dd47149c31f3c72d1ada7fadd19714d7270d18f71b52f1856298c56d7a"} Jul 21 13:51:03.620348 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.620245 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" event={"ID":"103cc13a-a074-47bf-aeac-68deb81f3520","Type":"ContainerStarted","Data":"4fca1af6f7c56c98c79a6526c7cdacee4ebf47d7b6475a6326e375fd120b9f66"} Jul 21 13:51:03.852806 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.852754 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb"] Jul 21 13:51:03.856230 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.856207 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" Jul 21 13:51:03.865787 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.865755 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb"] Jul 21 13:51:03.956642 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.956599 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/603b82ea-622a-461e-9114-4f787c689dbf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb\" (UID: \"603b82ea-622a-461e-9114-4f787c689dbf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" Jul 21 13:51:03.956642 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.956645 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxvgj\" (UniqueName: \"kubernetes.io/projected/603b82ea-622a-461e-9114-4f787c689dbf-kube-api-access-mxvgj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb\" (UID: \"603b82ea-622a-461e-9114-4f787c689dbf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" Jul 21 13:51:03.956887 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:03.956670 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/603b82ea-622a-461e-9114-4f787c689dbf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb\" (UID: \"603b82ea-622a-461e-9114-4f787c689dbf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" Jul 21 13:51:04.057179 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.057134 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/603b82ea-622a-461e-9114-4f787c689dbf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb\" (UID: \"603b82ea-622a-461e-9114-4f787c689dbf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" Jul 21 13:51:04.057179 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.057183 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxvgj\" (UniqueName: \"kubernetes.io/projected/603b82ea-622a-461e-9114-4f787c689dbf-kube-api-access-mxvgj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb\" (UID: \"603b82ea-622a-461e-9114-4f787c689dbf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" Jul 21 13:51:04.057626 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.057209 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/603b82ea-622a-461e-9114-4f787c689dbf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb\" (UID: \"603b82ea-622a-461e-9114-4f787c689dbf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" Jul 21 13:51:04.057626 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.057529 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/603b82ea-622a-461e-9114-4f787c689dbf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb\" (UID: \"603b82ea-622a-461e-9114-4f787c689dbf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" Jul 21 13:51:04.057626 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.057578 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/603b82ea-622a-461e-9114-4f787c689dbf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb\" (UID: \"603b82ea-622a-461e-9114-4f787c689dbf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" Jul 21 13:51:04.066624 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.066586 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxvgj\" (UniqueName: \"kubernetes.io/projected/603b82ea-622a-461e-9114-4f787c689dbf-kube-api-access-mxvgj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb\" (UID: \"603b82ea-622a-461e-9114-4f787c689dbf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" Jul 21 13:51:04.167361 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.167312 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" Jul 21 13:51:04.328596 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.328561 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb"] Jul 21 13:51:04.330490 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:51:04.330458 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod603b82ea_622a_461e_9114_4f787c689dbf.slice/crio-dee82309a0b7ea54f3d1bb7014c1b71072cbc77986b74605ac1023b1dd222182 WatchSource:0}: Error finding container dee82309a0b7ea54f3d1bb7014c1b71072cbc77986b74605ac1023b1dd222182: Status 404 returned error can't find the container with id dee82309a0b7ea54f3d1bb7014c1b71072cbc77986b74605ac1023b1dd222182 Jul 21 13:51:04.448013 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.447974 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9"] Jul 21 13:51:04.451788 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.451757 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" Jul 21 13:51:04.460355 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.460316 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9"] Jul 21 13:51:04.460639 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.460619 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b983d46-27cc-422c-9285-55da67e6c155-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9\" (UID: \"3b983d46-27cc-422c-9285-55da67e6c155\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" Jul 21 13:51:04.460693 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.460671 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b983d46-27cc-422c-9285-55da67e6c155-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9\" (UID: \"3b983d46-27cc-422c-9285-55da67e6c155\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" Jul 21 13:51:04.460774 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.460753 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fxbn\" (UniqueName: \"kubernetes.io/projected/3b983d46-27cc-422c-9285-55da67e6c155-kube-api-access-6fxbn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9\" (UID: \"3b983d46-27cc-422c-9285-55da67e6c155\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" Jul 21 13:51:04.561432 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.561378 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b983d46-27cc-422c-9285-55da67e6c155-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9\" (UID: \"3b983d46-27cc-422c-9285-55da67e6c155\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" Jul 21 13:51:04.561432 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.561422 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fxbn\" (UniqueName: \"kubernetes.io/projected/3b983d46-27cc-422c-9285-55da67e6c155-kube-api-access-6fxbn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9\" (UID: \"3b983d46-27cc-422c-9285-55da67e6c155\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" Jul 21 13:51:04.561665 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.561495 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b983d46-27cc-422c-9285-55da67e6c155-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9\" (UID: \"3b983d46-27cc-422c-9285-55da67e6c155\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" Jul 21 13:51:04.561856 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.561835 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b983d46-27cc-422c-9285-55da67e6c155-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9\" (UID: \"3b983d46-27cc-422c-9285-55da67e6c155\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" Jul 21 13:51:04.561905 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.561883 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b983d46-27cc-422c-9285-55da67e6c155-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9\" (UID: \"3b983d46-27cc-422c-9285-55da67e6c155\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" Jul 21 13:51:04.571953 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.571922 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fxbn\" (UniqueName: \"kubernetes.io/projected/3b983d46-27cc-422c-9285-55da67e6c155-kube-api-access-6fxbn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9\" (UID: \"3b983d46-27cc-422c-9285-55da67e6c155\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" Jul 21 13:51:04.629386 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.629321 2567 generic.go:356] "Generic (PLEG): container finished" podID="603b82ea-622a-461e-9114-4f787c689dbf" containerID="0d191590f56b992bf04bb26a2d0e762ba5f8dc52954b6361c2edd10eefdca304" exitCode=0 Jul 21 13:51:04.629839 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.629418 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" event={"ID":"603b82ea-622a-461e-9114-4f787c689dbf","Type":"ContainerDied","Data":"0d191590f56b992bf04bb26a2d0e762ba5f8dc52954b6361c2edd10eefdca304"} Jul 21 13:51:04.629839 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.629527 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" event={"ID":"603b82ea-622a-461e-9114-4f787c689dbf","Type":"ContainerStarted","Data":"dee82309a0b7ea54f3d1bb7014c1b71072cbc77986b74605ac1023b1dd222182"} Jul 21 13:51:04.632086 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.632040 2567 generic.go:356] "Generic (PLEG): container finished" podID="103cc13a-a074-47bf-aeac-68deb81f3520" containerID="13aa66dd47149c31f3c72d1ada7fadd19714d7270d18f71b52f1856298c56d7a" exitCode=0 Jul 21 13:51:04.632238 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.632100 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" event={"ID":"103cc13a-a074-47bf-aeac-68deb81f3520","Type":"ContainerDied","Data":"13aa66dd47149c31f3c72d1ada7fadd19714d7270d18f71b52f1856298c56d7a"} Jul 21 13:51:04.816501 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.816406 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" Jul 21 13:51:04.848958 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.848923 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6"] Jul 21 13:51:04.852648 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.852621 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" Jul 21 13:51:04.863861 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.863654 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6\" (UID: \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" Jul 21 13:51:04.863861 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.863754 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84wzg\" (UniqueName: \"kubernetes.io/projected/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-kube-api-access-84wzg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6\" (UID: \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" Jul 21 13:51:04.863861 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.863833 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6\" (UID: \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" Jul 21 13:51:04.865499 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.865470 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6"] Jul 21 13:51:04.964607 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.964571 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6\" (UID: \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" Jul 21 13:51:04.964799 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.964672 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6\" (UID: \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" Jul 21 13:51:04.964799 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.964748 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84wzg\" (UniqueName: \"kubernetes.io/projected/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-kube-api-access-84wzg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6\" (UID: \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" Jul 21 13:51:04.965032 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.965007 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6\" (UID: \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" Jul 21 13:51:04.965124 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.965056 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6\" (UID: \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" Jul 21 13:51:04.967147 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.967089 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9"] Jul 21 13:51:04.969340 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:51:04.969308 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b983d46_27cc_422c_9285_55da67e6c155.slice/crio-52b0e4a7820f5a53f919e23fe3ef8d502f21228da6c78f9717e77b59128e405b WatchSource:0}: Error finding container 52b0e4a7820f5a53f919e23fe3ef8d502f21228da6c78f9717e77b59128e405b: Status 404 returned error can't find the container with id 52b0e4a7820f5a53f919e23fe3ef8d502f21228da6c78f9717e77b59128e405b Jul 21 13:51:04.974967 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:04.974941 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84wzg\" (UniqueName: \"kubernetes.io/projected/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-kube-api-access-84wzg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6\" (UID: \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" Jul 21 13:51:05.164364 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:05.164318 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" Jul 21 13:51:05.320473 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:05.318791 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6"] Jul 21 13:51:05.639223 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:05.639187 2567 generic.go:356] "Generic (PLEG): container finished" podID="3b983d46-27cc-422c-9285-55da67e6c155" containerID="65e38ae3b22fa17ac0d79e3fb72e269629374cbaa4f6d735a57b2798a9005cdc" exitCode=0 Jul 21 13:51:05.639406 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:05.639260 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" event={"ID":"3b983d46-27cc-422c-9285-55da67e6c155","Type":"ContainerDied","Data":"65e38ae3b22fa17ac0d79e3fb72e269629374cbaa4f6d735a57b2798a9005cdc"} Jul 21 13:51:05.639406 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:05.639307 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" event={"ID":"3b983d46-27cc-422c-9285-55da67e6c155","Type":"ContainerStarted","Data":"52b0e4a7820f5a53f919e23fe3ef8d502f21228da6c78f9717e77b59128e405b"} Jul 21 13:51:05.640860 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:05.640830 2567 generic.go:356] "Generic (PLEG): container finished" podID="7d6bb23e-1c8c-4bc4-8edf-dc27214e9631" containerID="3d96553aa9095cd8a1b2011154a32d35fec3f6607f348da54492e737d6089bb1" exitCode=0 Jul 21 13:51:05.641000 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:05.640888 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" event={"ID":"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631","Type":"ContainerDied","Data":"3d96553aa9095cd8a1b2011154a32d35fec3f6607f348da54492e737d6089bb1"} Jul 21 13:51:05.641000 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:05.640930 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" event={"ID":"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631","Type":"ContainerStarted","Data":"5b9e23362351dd5edd8264f2243d9083ec76dd370aa07ad568ba84759f30b161"} Jul 21 13:51:06.649099 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:06.649064 2567 generic.go:356] "Generic (PLEG): container finished" podID="603b82ea-622a-461e-9114-4f787c689dbf" containerID="79a192dfd5f93e6281c67290e59c80b86ce34682f8312a6f7596f39e6cb4c028" exitCode=0 Jul 21 13:51:06.649495 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:06.649103 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" event={"ID":"603b82ea-622a-461e-9114-4f787c689dbf","Type":"ContainerDied","Data":"79a192dfd5f93e6281c67290e59c80b86ce34682f8312a6f7596f39e6cb4c028"} Jul 21 13:51:06.653634 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:06.653609 2567 generic.go:356] "Generic (PLEG): container finished" podID="103cc13a-a074-47bf-aeac-68deb81f3520" containerID="205e795917daf5bb5aa2edfc5260cbc29bd4615ae3232ceffa873b3403190d7d" exitCode=0 Jul 21 13:51:06.653791 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:06.653643 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" event={"ID":"103cc13a-a074-47bf-aeac-68deb81f3520","Type":"ContainerDied","Data":"205e795917daf5bb5aa2edfc5260cbc29bd4615ae3232ceffa873b3403190d7d"} Jul 21 13:51:07.663343 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:07.663300 2567 generic.go:356] "Generic (PLEG): container finished" podID="7d6bb23e-1c8c-4bc4-8edf-dc27214e9631" containerID="a617faa4cde1980f336797128872c060778e8d9526dd37d70b1efc34c2563e67" exitCode=0 Jul 21 13:51:07.663343 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:07.663344 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" event={"ID":"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631","Type":"ContainerDied","Data":"a617faa4cde1980f336797128872c060778e8d9526dd37d70b1efc34c2563e67"} Jul 21 13:51:07.665564 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:07.665538 2567 generic.go:356] "Generic (PLEG): container finished" podID="103cc13a-a074-47bf-aeac-68deb81f3520" containerID="a05a01d24f8d16b3b00eb0d4421cbcfa2f7f87082546a0b4468c9c0c13362fd8" exitCode=0 Jul 21 13:51:07.665659 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:07.665588 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" event={"ID":"103cc13a-a074-47bf-aeac-68deb81f3520","Type":"ContainerDied","Data":"a05a01d24f8d16b3b00eb0d4421cbcfa2f7f87082546a0b4468c9c0c13362fd8"} Jul 21 13:51:07.667809 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:07.667783 2567 generic.go:356] "Generic (PLEG): container finished" podID="603b82ea-622a-461e-9114-4f787c689dbf" containerID="3b66c592026df355ad112b78856717153153e1db33388b880891ff144a4a133d" exitCode=0 Jul 21 13:51:07.667908 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:07.667862 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" event={"ID":"603b82ea-622a-461e-9114-4f787c689dbf","Type":"ContainerDied","Data":"3b66c592026df355ad112b78856717153153e1db33388b880891ff144a4a133d"} Jul 21 13:51:07.669664 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:07.669643 2567 generic.go:356] "Generic (PLEG): container finished" podID="3b983d46-27cc-422c-9285-55da67e6c155" containerID="e3a6c59386efa85c39a03dcea31c830fc0da74ec73eea6aac3d0c68ee8d98848" exitCode=0 Jul 21 13:51:07.669779 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:07.669709 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" event={"ID":"3b983d46-27cc-422c-9285-55da67e6c155","Type":"ContainerDied","Data":"e3a6c59386efa85c39a03dcea31c830fc0da74ec73eea6aac3d0c68ee8d98848"} Jul 21 13:51:08.683055 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.683014 2567 generic.go:356] "Generic (PLEG): container finished" podID="3b983d46-27cc-422c-9285-55da67e6c155" containerID="aa1520c79d4db9b455b0faa4ca7bb7c951b88160b9d644e7706148e085626f2b" exitCode=0 Jul 21 13:51:08.683557 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.683095 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" event={"ID":"3b983d46-27cc-422c-9285-55da67e6c155","Type":"ContainerDied","Data":"aa1520c79d4db9b455b0faa4ca7bb7c951b88160b9d644e7706148e085626f2b"} Jul 21 13:51:08.685274 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.685246 2567 generic.go:356] "Generic (PLEG): container finished" podID="7d6bb23e-1c8c-4bc4-8edf-dc27214e9631" containerID="32a02b18c9f074b665d2698763c808e57d4f7c8a2a44db9b0f092ca468656fd0" exitCode=0 Jul 21 13:51:08.685419 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.685330 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" event={"ID":"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631","Type":"ContainerDied","Data":"32a02b18c9f074b665d2698763c808e57d4f7c8a2a44db9b0f092ca468656fd0"} Jul 21 13:51:08.803291 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.803093 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76b7c599df-jf44z"] Jul 21 13:51:08.808677 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.808646 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:08.825280 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.825128 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b7c599df-jf44z"] Jul 21 13:51:08.844173 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.844145 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" Jul 21 13:51:08.861349 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.861319 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" Jul 21 13:51:08.867277 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.867224 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:08.901250 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.901220 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/603b82ea-622a-461e-9114-4f787c689dbf-bundle\") pod \"603b82ea-622a-461e-9114-4f787c689dbf\" (UID: \"603b82ea-622a-461e-9114-4f787c689dbf\") " Jul 21 13:51:08.901432 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.901264 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dq6q\" (UniqueName: \"kubernetes.io/projected/103cc13a-a074-47bf-aeac-68deb81f3520-kube-api-access-9dq6q\") pod \"103cc13a-a074-47bf-aeac-68deb81f3520\" (UID: \"103cc13a-a074-47bf-aeac-68deb81f3520\") " Jul 21 13:51:08.901432 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.901301 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxvgj\" (UniqueName: \"kubernetes.io/projected/603b82ea-622a-461e-9114-4f787c689dbf-kube-api-access-mxvgj\") pod \"603b82ea-622a-461e-9114-4f787c689dbf\" (UID: \"603b82ea-622a-461e-9114-4f787c689dbf\") " Jul 21 13:51:08.901432 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.901327 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/603b82ea-622a-461e-9114-4f787c689dbf-util\") pod \"603b82ea-622a-461e-9114-4f787c689dbf\" (UID: \"603b82ea-622a-461e-9114-4f787c689dbf\") " Jul 21 13:51:08.901432 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.901366 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/103cc13a-a074-47bf-aeac-68deb81f3520-bundle\") pod \"103cc13a-a074-47bf-aeac-68deb81f3520\" (UID: \"103cc13a-a074-47bf-aeac-68deb81f3520\") " Jul 21 13:51:08.901723 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.901480 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/103cc13a-a074-47bf-aeac-68deb81f3520-util\") pod \"103cc13a-a074-47bf-aeac-68deb81f3520\" (UID: \"103cc13a-a074-47bf-aeac-68deb81f3520\") " Jul 21 13:51:08.901723 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.901635 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z4jv\" (UniqueName: \"kubernetes.io/projected/8ec89ad0-c63e-43cc-bbad-560d9e340689-kube-api-access-8z4jv\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:08.901723 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.901672 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ec89ad0-c63e-43cc-bbad-560d9e340689-console-config\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:08.901882 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.901757 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ec89ad0-c63e-43cc-bbad-560d9e340689-trusted-ca-bundle\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:08.901882 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.901853 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ec89ad0-c63e-43cc-bbad-560d9e340689-oauth-serving-cert\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:08.902059 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.902036 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ec89ad0-c63e-43cc-bbad-560d9e340689-console-serving-cert\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:08.902137 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.902094 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec89ad0-c63e-43cc-bbad-560d9e340689-service-ca\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:08.902137 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.902123 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ec89ad0-c63e-43cc-bbad-560d9e340689-console-oauth-config\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:08.902298 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.902275 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/603b82ea-622a-461e-9114-4f787c689dbf-bundle" (OuterVolumeSpecName: "bundle") pod "603b82ea-622a-461e-9114-4f787c689dbf" (UID: "603b82ea-622a-461e-9114-4f787c689dbf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:51:08.903915 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.903885 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/103cc13a-a074-47bf-aeac-68deb81f3520-bundle" (OuterVolumeSpecName: "bundle") pod "103cc13a-a074-47bf-aeac-68deb81f3520" (UID: "103cc13a-a074-47bf-aeac-68deb81f3520"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:51:08.904044 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.904024 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/603b82ea-622a-461e-9114-4f787c689dbf-kube-api-access-mxvgj" (OuterVolumeSpecName: "kube-api-access-mxvgj") pod "603b82ea-622a-461e-9114-4f787c689dbf" (UID: "603b82ea-622a-461e-9114-4f787c689dbf"). InnerVolumeSpecName "kube-api-access-mxvgj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:51:08.904121 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.904104 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/103cc13a-a074-47bf-aeac-68deb81f3520-kube-api-access-9dq6q" (OuterVolumeSpecName: "kube-api-access-9dq6q") pod "103cc13a-a074-47bf-aeac-68deb81f3520" (UID: "103cc13a-a074-47bf-aeac-68deb81f3520"). InnerVolumeSpecName "kube-api-access-9dq6q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:51:08.908586 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.908542 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/103cc13a-a074-47bf-aeac-68deb81f3520-util" (OuterVolumeSpecName: "util") pod "103cc13a-a074-47bf-aeac-68deb81f3520" (UID: "103cc13a-a074-47bf-aeac-68deb81f3520"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:51:08.909426 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.909292 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/603b82ea-622a-461e-9114-4f787c689dbf-util" (OuterVolumeSpecName: "util") pod "603b82ea-622a-461e-9114-4f787c689dbf" (UID: "603b82ea-622a-461e-9114-4f787c689dbf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:51:08.939134 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:08.939048 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:09.003043 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.003000 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ec89ad0-c63e-43cc-bbad-560d9e340689-console-serving-cert\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.003043 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.003044 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec89ad0-c63e-43cc-bbad-560d9e340689-service-ca\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.003300 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.003062 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ec89ad0-c63e-43cc-bbad-560d9e340689-console-oauth-config\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.003300 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.003084 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z4jv\" (UniqueName: \"kubernetes.io/projected/8ec89ad0-c63e-43cc-bbad-560d9e340689-kube-api-access-8z4jv\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.003300 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.003103 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ec89ad0-c63e-43cc-bbad-560d9e340689-console-config\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.003300 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.003128 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ec89ad0-c63e-43cc-bbad-560d9e340689-trusted-ca-bundle\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.003300 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.003162 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ec89ad0-c63e-43cc-bbad-560d9e340689-oauth-serving-cert\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.003300 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.003219 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/103cc13a-a074-47bf-aeac-68deb81f3520-util\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:09.003300 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.003235 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/603b82ea-622a-461e-9114-4f787c689dbf-bundle\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:09.003300 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.003248 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dq6q\" (UniqueName: \"kubernetes.io/projected/103cc13a-a074-47bf-aeac-68deb81f3520-kube-api-access-9dq6q\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:09.003300 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.003262 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxvgj\" (UniqueName: \"kubernetes.io/projected/603b82ea-622a-461e-9114-4f787c689dbf-kube-api-access-mxvgj\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:09.003300 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.003273 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/603b82ea-622a-461e-9114-4f787c689dbf-util\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:09.003300 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.003283 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/103cc13a-a074-47bf-aeac-68deb81f3520-bundle\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:09.004111 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.004086 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ec89ad0-c63e-43cc-bbad-560d9e340689-oauth-serving-cert\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.004185 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.004124 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ec89ad0-c63e-43cc-bbad-560d9e340689-console-config\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.004185 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.004092 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ec89ad0-c63e-43cc-bbad-560d9e340689-service-ca\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.004248 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.004206 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ec89ad0-c63e-43cc-bbad-560d9e340689-trusted-ca-bundle\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.005654 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.005622 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ec89ad0-c63e-43cc-bbad-560d9e340689-console-oauth-config\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.005801 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.005781 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ec89ad0-c63e-43cc-bbad-560d9e340689-console-serving-cert\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.012724 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.012685 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z4jv\" (UniqueName: \"kubernetes.io/projected/8ec89ad0-c63e-43cc-bbad-560d9e340689-kube-api-access-8z4jv\") pod \"console-76b7c599df-jf44z\" (UID: \"8ec89ad0-c63e-43cc-bbad-560d9e340689\") " pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.138971 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.138928 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:09.294605 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.294575 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b7c599df-jf44z"] Jul 21 13:51:09.296003 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:51:09.295975 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec89ad0_c63e_43cc_bbad_560d9e340689.slice/crio-78571f9fecee04dd10f05210f44e97b948b2319e56278ca8e93be03646fb69f6 WatchSource:0}: Error finding container 78571f9fecee04dd10f05210f44e97b948b2319e56278ca8e93be03646fb69f6: Status 404 returned error can't find the container with id 78571f9fecee04dd10f05210f44e97b948b2319e56278ca8e93be03646fb69f6 Jul 21 13:51:09.692792 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.692758 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" Jul 21 13:51:09.693229 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.692763 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb" event={"ID":"603b82ea-622a-461e-9114-4f787c689dbf","Type":"ContainerDied","Data":"dee82309a0b7ea54f3d1bb7014c1b71072cbc77986b74605ac1023b1dd222182"} Jul 21 13:51:09.693229 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.692885 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dee82309a0b7ea54f3d1bb7014c1b71072cbc77986b74605ac1023b1dd222182" Jul 21 13:51:09.694528 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.694503 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b7c599df-jf44z" event={"ID":"8ec89ad0-c63e-43cc-bbad-560d9e340689","Type":"ContainerStarted","Data":"03bb8e8d6b1ca9096cc7d97a6a1cbe029bb2cf096c3ed2c64406be853df86dcb"} Jul 21 13:51:09.694685 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.694546 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b7c599df-jf44z" event={"ID":"8ec89ad0-c63e-43cc-bbad-560d9e340689","Type":"ContainerStarted","Data":"78571f9fecee04dd10f05210f44e97b948b2319e56278ca8e93be03646fb69f6"} Jul 21 13:51:09.696617 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.696583 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" event={"ID":"103cc13a-a074-47bf-aeac-68deb81f3520","Type":"ContainerDied","Data":"4fca1af6f7c56c98c79a6526c7cdacee4ebf47d7b6475a6326e375fd120b9f66"} Jul 21 13:51:09.696617 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.696606 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv" Jul 21 13:51:09.696796 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.696627 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fca1af6f7c56c98c79a6526c7cdacee4ebf47d7b6475a6326e375fd120b9f66" Jul 21 13:51:09.716100 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.716021 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76b7c599df-jf44z" podStartSLOduration=1.7159996720000001 podStartE2EDuration="1.715999672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:51:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:51:09.71382602 +0000 UTC m=+430.865667122" watchObservedRunningTime="2026-07-21 13:51:09.715999672 +0000 UTC m=+430.867840782" Jul 21 13:51:09.853192 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.853163 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" Jul 21 13:51:09.869261 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.869237 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" Jul 21 13:51:09.918432 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.918389 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b983d46-27cc-422c-9285-55da67e6c155-util\") pod \"3b983d46-27cc-422c-9285-55da67e6c155\" (UID: \"3b983d46-27cc-422c-9285-55da67e6c155\") " Jul 21 13:51:09.918637 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.918454 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-bundle\") pod \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\" (UID: \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\") " Jul 21 13:51:09.918637 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.918504 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fxbn\" (UniqueName: \"kubernetes.io/projected/3b983d46-27cc-422c-9285-55da67e6c155-kube-api-access-6fxbn\") pod \"3b983d46-27cc-422c-9285-55da67e6c155\" (UID: \"3b983d46-27cc-422c-9285-55da67e6c155\") " Jul 21 13:51:09.919104 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.919078 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b983d46-27cc-422c-9285-55da67e6c155-bundle\") pod \"3b983d46-27cc-422c-9285-55da67e6c155\" (UID: \"3b983d46-27cc-422c-9285-55da67e6c155\") " Jul 21 13:51:09.919203 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.919144 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84wzg\" (UniqueName: \"kubernetes.io/projected/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-kube-api-access-84wzg\") pod \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\" (UID: \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\") " Jul 21 13:51:09.919203 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.919182 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-util\") pod \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\" (UID: \"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631\") " Jul 21 13:51:09.919317 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.919231 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-bundle" (OuterVolumeSpecName: "bundle") pod "7d6bb23e-1c8c-4bc4-8edf-dc27214e9631" (UID: "7d6bb23e-1c8c-4bc4-8edf-dc27214e9631"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:51:09.919921 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.919471 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-bundle\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:09.920201 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.920173 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b983d46-27cc-422c-9285-55da67e6c155-bundle" (OuterVolumeSpecName: "bundle") pod "3b983d46-27cc-422c-9285-55da67e6c155" (UID: "3b983d46-27cc-422c-9285-55da67e6c155"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:51:09.921614 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.921590 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-kube-api-access-84wzg" (OuterVolumeSpecName: "kube-api-access-84wzg") pod "7d6bb23e-1c8c-4bc4-8edf-dc27214e9631" (UID: "7d6bb23e-1c8c-4bc4-8edf-dc27214e9631"). InnerVolumeSpecName "kube-api-access-84wzg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:51:09.922419 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.922398 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b983d46-27cc-422c-9285-55da67e6c155-kube-api-access-6fxbn" (OuterVolumeSpecName: "kube-api-access-6fxbn") pod "3b983d46-27cc-422c-9285-55da67e6c155" (UID: "3b983d46-27cc-422c-9285-55da67e6c155"). InnerVolumeSpecName "kube-api-access-6fxbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:51:09.925474 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.925450 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b983d46-27cc-422c-9285-55da67e6c155-util" (OuterVolumeSpecName: "util") pod "3b983d46-27cc-422c-9285-55da67e6c155" (UID: "3b983d46-27cc-422c-9285-55da67e6c155"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:51:09.925590 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:09.925570 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-util" (OuterVolumeSpecName: "util") pod "7d6bb23e-1c8c-4bc4-8edf-dc27214e9631" (UID: "7d6bb23e-1c8c-4bc4-8edf-dc27214e9631"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:51:10.020351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:10.020312 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fxbn\" (UniqueName: \"kubernetes.io/projected/3b983d46-27cc-422c-9285-55da67e6c155-kube-api-access-6fxbn\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:10.020351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:10.020345 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3b983d46-27cc-422c-9285-55da67e6c155-bundle\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:10.020351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:10.020356 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84wzg\" (UniqueName: \"kubernetes.io/projected/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-kube-api-access-84wzg\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:10.020581 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:10.020365 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d6bb23e-1c8c-4bc4-8edf-dc27214e9631-util\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:10.020581 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:10.020375 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3b983d46-27cc-422c-9285-55da67e6c155-util\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:10.703460 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:10.703428 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" Jul 21 13:51:10.703906 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:10.703434 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6" event={"ID":"7d6bb23e-1c8c-4bc4-8edf-dc27214e9631","Type":"ContainerDied","Data":"5b9e23362351dd5edd8264f2243d9083ec76dd370aa07ad568ba84759f30b161"} Jul 21 13:51:10.703906 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:10.703579 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b9e23362351dd5edd8264f2243d9083ec76dd370aa07ad568ba84759f30b161" Jul 21 13:51:10.705657 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:10.705635 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" Jul 21 13:51:10.705824 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:10.705672 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9" event={"ID":"3b983d46-27cc-422c-9285-55da67e6c155","Type":"ContainerDied","Data":"52b0e4a7820f5a53f919e23fe3ef8d502f21228da6c78f9717e77b59128e405b"} Jul 21 13:51:10.705824 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:10.705746 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52b0e4a7820f5a53f919e23fe3ef8d502f21228da6c78f9717e77b59128e405b" Jul 21 13:51:12.071415 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:12.071378 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:17.195469 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:17.195431 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:19.139521 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.139473 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:19.139965 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.139554 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:19.144736 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.144691 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:19.754770 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.754742 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76b7c599df-jf44z" Jul 21 13:51:19.826260 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.826228 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6997c89b48-hcdmm"] Jul 21 13:51:19.849035 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849003 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf"] Jul 21 13:51:19.849529 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849510 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="103cc13a-a074-47bf-aeac-68deb81f3520" containerName="extract" Jul 21 13:51:19.849576 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849534 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="103cc13a-a074-47bf-aeac-68deb81f3520" containerName="extract" Jul 21 13:51:19.849576 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849549 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6bb23e-1c8c-4bc4-8edf-dc27214e9631" containerName="pull" Jul 21 13:51:19.849576 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849558 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6bb23e-1c8c-4bc4-8edf-dc27214e9631" containerName="pull" Jul 21 13:51:19.849576 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849573 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="103cc13a-a074-47bf-aeac-68deb81f3520" containerName="util" Jul 21 13:51:19.849690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849582 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="103cc13a-a074-47bf-aeac-68deb81f3520" containerName="util" Jul 21 13:51:19.849690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849592 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="603b82ea-622a-461e-9114-4f787c689dbf" containerName="pull" Jul 21 13:51:19.849690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849600 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="603b82ea-622a-461e-9114-4f787c689dbf" containerName="pull" Jul 21 13:51:19.849690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849611 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="603b82ea-622a-461e-9114-4f787c689dbf" containerName="extract" Jul 21 13:51:19.849690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849620 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="603b82ea-622a-461e-9114-4f787c689dbf" containerName="extract" Jul 21 13:51:19.849690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849631 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="603b82ea-622a-461e-9114-4f787c689dbf" containerName="util" Jul 21 13:51:19.849690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849639 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="603b82ea-622a-461e-9114-4f787c689dbf" containerName="util" Jul 21 13:51:19.849690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849649 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b983d46-27cc-422c-9285-55da67e6c155" containerName="pull" Jul 21 13:51:19.849690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849657 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b983d46-27cc-422c-9285-55da67e6c155" containerName="pull" Jul 21 13:51:19.849690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849675 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6bb23e-1c8c-4bc4-8edf-dc27214e9631" containerName="util" Jul 21 13:51:19.849690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849684 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6bb23e-1c8c-4bc4-8edf-dc27214e9631" containerName="util" Jul 21 13:51:19.849690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849693 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6bb23e-1c8c-4bc4-8edf-dc27214e9631" containerName="extract" Jul 21 13:51:19.850040 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849719 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6bb23e-1c8c-4bc4-8edf-dc27214e9631" containerName="extract" Jul 21 13:51:19.850040 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849732 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b983d46-27cc-422c-9285-55da67e6c155" containerName="util" Jul 21 13:51:19.850040 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849740 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b983d46-27cc-422c-9285-55da67e6c155" containerName="util" Jul 21 13:51:19.850040 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849753 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b983d46-27cc-422c-9285-55da67e6c155" containerName="extract" Jul 21 13:51:19.850040 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849761 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b983d46-27cc-422c-9285-55da67e6c155" containerName="extract" Jul 21 13:51:19.850040 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849777 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="103cc13a-a074-47bf-aeac-68deb81f3520" containerName="pull" Jul 21 13:51:19.850040 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849785 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="103cc13a-a074-47bf-aeac-68deb81f3520" containerName="pull" Jul 21 13:51:19.850040 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849879 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b983d46-27cc-422c-9285-55da67e6c155" containerName="extract" Jul 21 13:51:19.850040 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849891 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d6bb23e-1c8c-4bc4-8edf-dc27214e9631" containerName="extract" Jul 21 13:51:19.850040 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849900 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="103cc13a-a074-47bf-aeac-68deb81f3520" containerName="extract" Jul 21 13:51:19.850040 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.849913 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="603b82ea-622a-461e-9114-4f787c689dbf" containerName="extract" Jul 21 13:51:19.854570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.854548 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" Jul 21 13:51:19.857997 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.857850 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-tdgc6\"" Jul 21 13:51:19.869351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.869324 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf"] Jul 21 13:51:19.913608 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:19.913569 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzrlc\" (UniqueName: \"kubernetes.io/projected/570dd1e5-86c8-4877-9e8b-ed536ce227df-kube-api-access-xzrlc\") pod \"limitador-operator-controller-manager-859d7b78bd-qnzmf\" (UID: \"570dd1e5-86c8-4877-9e8b-ed536ce227df\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" Jul 21 13:51:20.014785 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:20.014667 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzrlc\" (UniqueName: \"kubernetes.io/projected/570dd1e5-86c8-4877-9e8b-ed536ce227df-kube-api-access-xzrlc\") pod \"limitador-operator-controller-manager-859d7b78bd-qnzmf\" (UID: \"570dd1e5-86c8-4877-9e8b-ed536ce227df\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" Jul 21 13:51:20.025305 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:20.025276 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzrlc\" (UniqueName: \"kubernetes.io/projected/570dd1e5-86c8-4877-9e8b-ed536ce227df-kube-api-access-xzrlc\") pod \"limitador-operator-controller-manager-859d7b78bd-qnzmf\" (UID: \"570dd1e5-86c8-4877-9e8b-ed536ce227df\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" Jul 21 13:51:20.166176 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:20.166135 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" Jul 21 13:51:20.303161 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:20.303073 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf"] Jul 21 13:51:20.306320 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:51:20.306291 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570dd1e5_86c8_4877_9e8b_ed536ce227df.slice/crio-1c5f593f8764c91c9554af877f65b71bffe3b9d6bb237c9c108cbec5f53504b4 WatchSource:0}: Error finding container 1c5f593f8764c91c9554af877f65b71bffe3b9d6bb237c9c108cbec5f53504b4: Status 404 returned error can't find the container with id 1c5f593f8764c91c9554af877f65b71bffe3b9d6bb237c9c108cbec5f53504b4 Jul 21 13:51:20.505460 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:20.505422 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:20.756794 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:20.756754 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" event={"ID":"570dd1e5-86c8-4877-9e8b-ed536ce227df","Type":"ContainerStarted","Data":"1c5f593f8764c91c9554af877f65b71bffe3b9d6bb237c9c108cbec5f53504b4"} Jul 21 13:51:21.028882 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:21.028791 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:21.126899 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:21.126859 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:21.422625 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:21.422574 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:23.776228 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:23.776191 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" event={"ID":"570dd1e5-86c8-4877-9e8b-ed536ce227df","Type":"ContainerStarted","Data":"348bc59be7050f21a1cefc476fa94c93f55be30646bb423373c966774203d767"} Jul 21 13:51:23.776750 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:23.776265 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" Jul 21 13:51:23.803952 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:23.803895 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" podStartSLOduration=2.402383732 podStartE2EDuration="4.803879226s" totalImagesPullingTime="2.401495494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:51:23.801060599 +0000 UTC m=+444.952901710" watchObservedRunningTime="2026-07-21 13:51:23.803879226 +0000 UTC m=+444.955720334" Jul 21 13:51:25.138005 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:25.137958 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:25.236532 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:25.236486 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:26.890089 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:26.890053 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:27.238151 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:27.238118 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-d4qgk"] Jul 21 13:51:27.245881 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:27.245852 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d4qgk" Jul 21 13:51:27.248681 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:27.248658 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 21 13:51:27.249116 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:27.249094 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-2l478\"" Jul 21 13:51:27.260839 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:27.260796 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-d4qgk"] Jul 21 13:51:27.382912 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:27.382866 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkh6d\" (UniqueName: \"kubernetes.io/projected/5110a6b1-b898-43c4-8229-62dcb157f267-kube-api-access-tkh6d\") pod \"dns-operator-controller-manager-864dd78c4c-d4qgk\" (UID: \"5110a6b1-b898-43c4-8229-62dcb157f267\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d4qgk" Jul 21 13:51:27.484313 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:27.484275 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkh6d\" (UniqueName: \"kubernetes.io/projected/5110a6b1-b898-43c4-8229-62dcb157f267-kube-api-access-tkh6d\") pod \"dns-operator-controller-manager-864dd78c4c-d4qgk\" (UID: \"5110a6b1-b898-43c4-8229-62dcb157f267\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d4qgk" Jul 21 13:51:27.497439 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:27.497370 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkh6d\" (UniqueName: \"kubernetes.io/projected/5110a6b1-b898-43c4-8229-62dcb157f267-kube-api-access-tkh6d\") pod \"dns-operator-controller-manager-864dd78c4c-d4qgk\" (UID: \"5110a6b1-b898-43c4-8229-62dcb157f267\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d4qgk" Jul 21 13:51:27.557327 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:27.557287 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d4qgk" Jul 21 13:51:27.749372 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:27.749341 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-d4qgk"] Jul 21 13:51:27.750778 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:51:27.750747 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5110a6b1_b898_43c4_8229_62dcb157f267.slice/crio-d76f09d32157f84372161d15aa4024c9a55d739851e96c9bbd417ad85e954ed0 WatchSource:0}: Error finding container d76f09d32157f84372161d15aa4024c9a55d739851e96c9bbd417ad85e954ed0: Status 404 returned error can't find the container with id d76f09d32157f84372161d15aa4024c9a55d739851e96c9bbd417ad85e954ed0 Jul 21 13:51:27.797981 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:27.797943 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d4qgk" event={"ID":"5110a6b1-b898-43c4-8229-62dcb157f267","Type":"ContainerStarted","Data":"d76f09d32157f84372161d15aa4024c9a55d739851e96c9bbd417ad85e954ed0"} Jul 21 13:51:27.837253 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:27.837213 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:29.051256 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:29.051205 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:30.815886 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:30.815851 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d4qgk" event={"ID":"5110a6b1-b898-43c4-8229-62dcb157f267","Type":"ContainerStarted","Data":"13f5e52925abd7d151af0be4cd5ca4d4be7f43cb8f874263ac70f5f76373eb3a"} Jul 21 13:51:30.816293 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:30.815920 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d4qgk" Jul 21 13:51:30.837516 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:30.837457 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d4qgk" podStartSLOduration=1.40743711 podStartE2EDuration="3.837440348s" totalImagesPullingTime="2.430003238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:51:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:51:30.834921222 +0000 UTC m=+451.986762344" watchObservedRunningTime="2026-07-21 13:51:30.837440348 +0000 UTC m=+451.989281457" Jul 21 13:51:32.277844 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:32.277799 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:32.490936 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:32.490395 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv"] Jul 21 13:51:32.495566 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:32.495534 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:32.500183 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:32.500137 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-44l6z\"" Jul 21 13:51:32.518231 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:32.518198 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv"] Jul 21 13:51:32.632893 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:32.632787 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v5t7\" (UniqueName: \"kubernetes.io/projected/6a655f0e-b533-471c-b036-e45ff86bc2d3-kube-api-access-6v5t7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l7pnv\" (UID: \"6a655f0e-b533-471c-b036-e45ff86bc2d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:32.633082 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:32.632934 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a655f0e-b533-471c-b036-e45ff86bc2d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l7pnv\" (UID: \"6a655f0e-b533-471c-b036-e45ff86bc2d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:32.734065 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:32.734024 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a655f0e-b533-471c-b036-e45ff86bc2d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l7pnv\" (UID: \"6a655f0e-b533-471c-b036-e45ff86bc2d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:32.734244 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:32.734097 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v5t7\" (UniqueName: \"kubernetes.io/projected/6a655f0e-b533-471c-b036-e45ff86bc2d3-kube-api-access-6v5t7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l7pnv\" (UID: \"6a655f0e-b533-471c-b036-e45ff86bc2d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:32.734516 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:32.734493 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a655f0e-b533-471c-b036-e45ff86bc2d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l7pnv\" (UID: \"6a655f0e-b533-471c-b036-e45ff86bc2d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:32.744130 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:32.744100 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v5t7\" (UniqueName: \"kubernetes.io/projected/6a655f0e-b533-471c-b036-e45ff86bc2d3-kube-api-access-6v5t7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l7pnv\" (UID: \"6a655f0e-b533-471c-b036-e45ff86bc2d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:32.807273 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:32.807214 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:32.952196 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:32.952169 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv"] Jul 21 13:51:32.953150 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:51:32.953126 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a655f0e_b533_471c_b036_e45ff86bc2d3.slice/crio-ef8b9ec4ab86d7ba4791e2df9cf76a9dc233b9e831ccc70815c052d3b177122d WatchSource:0}: Error finding container ef8b9ec4ab86d7ba4791e2df9cf76a9dc233b9e831ccc70815c052d3b177122d: Status 404 returned error can't find the container with id ef8b9ec4ab86d7ba4791e2df9cf76a9dc233b9e831ccc70815c052d3b177122d Jul 21 13:51:33.814742 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:33.814674 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:33.835872 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:33.835827 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" event={"ID":"6a655f0e-b533-471c-b036-e45ff86bc2d3","Type":"ContainerStarted","Data":"ef8b9ec4ab86d7ba4791e2df9cf76a9dc233b9e831ccc70815c052d3b177122d"} Jul 21 13:51:33.920274 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:33.920233 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:34.784439 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:34.784411 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" Jul 21 13:51:37.864293 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:37.864254 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" event={"ID":"6a655f0e-b533-471c-b036-e45ff86bc2d3","Type":"ContainerStarted","Data":"a8f5e4ecf8f71e5df9c5ec6e96b9e0bb9d240a17dcf48649cfea1728bca52a69"} Jul 21 13:51:37.864756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:37.864444 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:37.889983 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:37.889870 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" podStartSLOduration=1.208217944 podStartE2EDuration="5.889848804s" totalImagesPullingTime="4.68163086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:51:37.888326496 +0000 UTC m=+459.040167595" watchObservedRunningTime="2026-07-21 13:51:37.889848804 +0000 UTC m=+459.041689915" Jul 21 13:51:37.937153 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:37.937116 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:38.017243 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:38.017214 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:38.264000 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:38.263968 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:38.354801 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:38.354770 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:40.396310 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:40.396270 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:40.473026 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:40.472992 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:41.197574 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:41.197538 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:41.298003 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:41.297966 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:41.824398 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:41.824365 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-d4qgk" Jul 21 13:51:44.849184 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:44.849138 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6997c89b48-hcdmm" podUID="e0d58ce6-e413-4350-8a19-3df19927dda5" containerName="console" containerID="cri-o://ca2c95a08e77284063b6e59437a599e26b18b1e68ad391d578ec2ab348ec2677" gracePeriod=15 Jul 21 13:51:45.105037 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.104955 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:51:45.218421 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.218387 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:45.253528 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.253493 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-service-ca\") pod \"e0d58ce6-e413-4350-8a19-3df19927dda5\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " Jul 21 13:51:45.253690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.253544 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-oauth-serving-cert\") pod \"e0d58ce6-e413-4350-8a19-3df19927dda5\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " Jul 21 13:51:45.253690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.253580 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0d58ce6-e413-4350-8a19-3df19927dda5-console-serving-cert\") pod \"e0d58ce6-e413-4350-8a19-3df19927dda5\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " Jul 21 13:51:45.253690 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.253661 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-trusted-ca-bundle\") pod \"e0d58ce6-e413-4350-8a19-3df19927dda5\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " Jul 21 13:51:45.253882 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.253694 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btmdh\" (UniqueName: \"kubernetes.io/projected/e0d58ce6-e413-4350-8a19-3df19927dda5-kube-api-access-btmdh\") pod \"e0d58ce6-e413-4350-8a19-3df19927dda5\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " Jul 21 13:51:45.253882 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.253760 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-console-config\") pod \"e0d58ce6-e413-4350-8a19-3df19927dda5\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " Jul 21 13:51:45.253882 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.253825 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0d58ce6-e413-4350-8a19-3df19927dda5-console-oauth-config\") pod \"e0d58ce6-e413-4350-8a19-3df19927dda5\" (UID: \"e0d58ce6-e413-4350-8a19-3df19927dda5\") " Jul 21 13:51:45.254337 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.254286 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e0d58ce6-e413-4350-8a19-3df19927dda5" (UID: "e0d58ce6-e413-4350-8a19-3df19927dda5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:51:45.254480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.254360 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-service-ca" (OuterVolumeSpecName: "service-ca") pod "e0d58ce6-e413-4350-8a19-3df19927dda5" (UID: "e0d58ce6-e413-4350-8a19-3df19927dda5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:51:45.256514 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.254741 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-console-config" (OuterVolumeSpecName: "console-config") pod "e0d58ce6-e413-4350-8a19-3df19927dda5" (UID: "e0d58ce6-e413-4350-8a19-3df19927dda5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:51:45.256514 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.255097 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e0d58ce6-e413-4350-8a19-3df19927dda5" (UID: "e0d58ce6-e413-4350-8a19-3df19927dda5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:51:45.256682 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.256646 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0d58ce6-e413-4350-8a19-3df19927dda5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e0d58ce6-e413-4350-8a19-3df19927dda5" (UID: "e0d58ce6-e413-4350-8a19-3df19927dda5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:51:45.263073 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.263044 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0d58ce6-e413-4350-8a19-3df19927dda5-kube-api-access-btmdh" (OuterVolumeSpecName: "kube-api-access-btmdh") pod "e0d58ce6-e413-4350-8a19-3df19927dda5" (UID: "e0d58ce6-e413-4350-8a19-3df19927dda5"). InnerVolumeSpecName "kube-api-access-btmdh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:51:45.264178 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.264032 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0d58ce6-e413-4350-8a19-3df19927dda5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e0d58ce6-e413-4350-8a19-3df19927dda5" (UID: "e0d58ce6-e413-4350-8a19-3df19927dda5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:51:45.319885 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.319847 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:45.355858 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.355778 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0d58ce6-e413-4350-8a19-3df19927dda5-console-oauth-config\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:45.355858 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.355805 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-service-ca\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:45.355858 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.355815 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-oauth-serving-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:45.355858 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.355823 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0d58ce6-e413-4350-8a19-3df19927dda5-console-serving-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:45.355858 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.355832 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-trusted-ca-bundle\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:45.355858 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.355841 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btmdh\" (UniqueName: \"kubernetes.io/projected/e0d58ce6-e413-4350-8a19-3df19927dda5-kube-api-access-btmdh\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:45.355858 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.355850 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0d58ce6-e413-4350-8a19-3df19927dda5-console-config\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:45.907134 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.907100 2567 generic.go:356] "Generic (PLEG): container finished" podID="e0d58ce6-e413-4350-8a19-3df19927dda5" containerID="ca2c95a08e77284063b6e59437a599e26b18b1e68ad391d578ec2ab348ec2677" exitCode=0 Jul 21 13:51:45.907565 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.907155 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6997c89b48-hcdmm" Jul 21 13:51:45.907565 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.907164 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6997c89b48-hcdmm" event={"ID":"e0d58ce6-e413-4350-8a19-3df19927dda5","Type":"ContainerDied","Data":"ca2c95a08e77284063b6e59437a599e26b18b1e68ad391d578ec2ab348ec2677"} Jul 21 13:51:45.907565 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.907208 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6997c89b48-hcdmm" event={"ID":"e0d58ce6-e413-4350-8a19-3df19927dda5","Type":"ContainerDied","Data":"f522933aa6edf1eb9f66040381823647baaa44d4e0cdcdc3bda129cf964d287b"} Jul 21 13:51:45.907565 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.907229 2567 scope.go:117] "RemoveContainer" containerID="ca2c95a08e77284063b6e59437a599e26b18b1e68ad391d578ec2ab348ec2677" Jul 21 13:51:45.917455 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.917434 2567 scope.go:117] "RemoveContainer" containerID="ca2c95a08e77284063b6e59437a599e26b18b1e68ad391d578ec2ab348ec2677" Jul 21 13:51:45.917749 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:51:45.917722 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca2c95a08e77284063b6e59437a599e26b18b1e68ad391d578ec2ab348ec2677\": container with ID starting with ca2c95a08e77284063b6e59437a599e26b18b1e68ad391d578ec2ab348ec2677 not found: ID does not exist" containerID="ca2c95a08e77284063b6e59437a599e26b18b1e68ad391d578ec2ab348ec2677" Jul 21 13:51:45.917827 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.917759 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca2c95a08e77284063b6e59437a599e26b18b1e68ad391d578ec2ab348ec2677"} err="failed to get container status \"ca2c95a08e77284063b6e59437a599e26b18b1e68ad391d578ec2ab348ec2677\": rpc error: code = NotFound desc = could not find container \"ca2c95a08e77284063b6e59437a599e26b18b1e68ad391d578ec2ab348ec2677\": container with ID starting with ca2c95a08e77284063b6e59437a599e26b18b1e68ad391d578ec2ab348ec2677 not found: ID does not exist" Jul 21 13:51:45.936470 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.936433 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6997c89b48-hcdmm"] Jul 21 13:51:45.941301 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:45.941277 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6997c89b48-hcdmm"] Jul 21 13:51:47.533140 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:47.533105 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0d58ce6-e413-4350-8a19-3df19927dda5" path="/var/lib/kubelet/pods/e0d58ce6-e413-4350-8a19-3df19927dda5/volumes" Jul 21 13:51:48.873112 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:48.873079 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:50.706367 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.706326 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6"] Jul 21 13:51:50.706767 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.706675 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d58ce6-e413-4350-8a19-3df19927dda5" containerName="console" Jul 21 13:51:50.706767 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.706686 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d58ce6-e413-4350-8a19-3df19927dda5" containerName="console" Jul 21 13:51:50.706847 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.706769 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d58ce6-e413-4350-8a19-3df19927dda5" containerName="console" Jul 21 13:51:50.711370 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.711350 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" Jul 21 13:51:50.741553 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.741508 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6"] Jul 21 13:51:50.769736 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.767963 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv"] Jul 21 13:51:50.769736 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.768311 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" podUID="6a655f0e-b533-471c-b036-e45ff86bc2d3" containerName="manager" containerID="cri-o://a8f5e4ecf8f71e5df9c5ec6e96b9e0bb9d240a17dcf48649cfea1728bca52a69" gracePeriod=2 Jul 21 13:51:50.777527 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:51:50.777492 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7pnv\" is forbidden: User \"system:node:ip-10-0-129-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-212.ec2.internal' and this object" podUID="6a655f0e-b533-471c-b036-e45ff86bc2d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:50.780206 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.780179 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv"] Jul 21 13:51:50.798291 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.798254 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf"] Jul 21 13:51:50.798589 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.798534 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" podUID="570dd1e5-86c8-4877-9e8b-ed536ce227df" containerName="manager" containerID="cri-o://348bc59be7050f21a1cefc476fa94c93f55be30646bb423373c966774203d767" gracePeriod=2 Jul 21 13:51:50.802686 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:51:50.802646 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7pnv\" is forbidden: User \"system:node:ip-10-0-129-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-212.ec2.internal' and this object" podUID="6a655f0e-b533-471c-b036-e45ff86bc2d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:50.802839 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.802815 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1848a5d0-54ec-4255-a1ef-4a2f0a7bd059-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kb2q6\" (UID: \"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" Jul 21 13:51:50.802905 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.802863 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lg8v\" (UniqueName: \"kubernetes.io/projected/1848a5d0-54ec-4255-a1ef-4a2f0a7bd059-kube-api-access-2lg8v\") pod \"kuadrant-operator-controller-manager-86878c769f-kb2q6\" (UID: \"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" Jul 21 13:51:50.813591 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.813556 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf"] Jul 21 13:51:50.813767 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:51:50.813739 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnzmf\" is forbidden: User \"system:node:ip-10-0-129-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-212.ec2.internal' and this object" podUID="570dd1e5-86c8-4877-9e8b-ed536ce227df" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" Jul 21 13:51:50.844953 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.844916 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jpbms"] Jul 21 13:51:50.845410 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.845391 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="570dd1e5-86c8-4877-9e8b-ed536ce227df" containerName="manager" Jul 21 13:51:50.845410 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.845412 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="570dd1e5-86c8-4877-9e8b-ed536ce227df" containerName="manager" Jul 21 13:51:50.845560 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.845441 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a655f0e-b533-471c-b036-e45ff86bc2d3" containerName="manager" Jul 21 13:51:50.845560 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.845449 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a655f0e-b533-471c-b036-e45ff86bc2d3" containerName="manager" Jul 21 13:51:50.845560 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.845547 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a655f0e-b533-471c-b036-e45ff86bc2d3" containerName="manager" Jul 21 13:51:50.845560 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.845558 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="570dd1e5-86c8-4877-9e8b-ed536ce227df" containerName="manager" Jul 21 13:51:50.849019 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.848991 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jpbms" Jul 21 13:51:50.851600 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:51:50.851553 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7pnv\" is forbidden: User \"system:node:ip-10-0-129-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-212.ec2.internal' and this object" podUID="6a655f0e-b533-471c-b036-e45ff86bc2d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:50.854016 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:51:50.853986 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnzmf\" is forbidden: User \"system:node:ip-10-0-129-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-212.ec2.internal' and this object" podUID="570dd1e5-86c8-4877-9e8b-ed536ce227df" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" Jul 21 13:51:50.871440 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.871397 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jpbms"] Jul 21 13:51:50.892606 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:51:50.892571 2567 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570dd1e5_86c8_4877_9e8b_ed536ce227df.slice/crio-conmon-348bc59be7050f21a1cefc476fa94c93f55be30646bb423373c966774203d767.scope\": RecentStats: unable to find data in memory cache]" Jul 21 13:51:50.903759 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.903724 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1848a5d0-54ec-4255-a1ef-4a2f0a7bd059-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kb2q6\" (UID: \"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" Jul 21 13:51:50.903917 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.903778 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lg8v\" (UniqueName: \"kubernetes.io/projected/1848a5d0-54ec-4255-a1ef-4a2f0a7bd059-kube-api-access-2lg8v\") pod \"kuadrant-operator-controller-manager-86878c769f-kb2q6\" (UID: \"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" Jul 21 13:51:50.903917 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.903813 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r77z\" (UniqueName: \"kubernetes.io/projected/5f4b4477-39a1-4f98-9d08-d70bcffb489a-kube-api-access-9r77z\") pod \"limitador-operator-controller-manager-859d7b78bd-jpbms\" (UID: \"5f4b4477-39a1-4f98-9d08-d70bcffb489a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jpbms" Jul 21 13:51:50.904143 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.904118 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1848a5d0-54ec-4255-a1ef-4a2f0a7bd059-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kb2q6\" (UID: \"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" Jul 21 13:51:50.915516 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.915468 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lg8v\" (UniqueName: \"kubernetes.io/projected/1848a5d0-54ec-4255-a1ef-4a2f0a7bd059-kube-api-access-2lg8v\") pod \"kuadrant-operator-controller-manager-86878c769f-kb2q6\" (UID: \"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" Jul 21 13:51:50.940436 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.940375 2567 generic.go:356] "Generic (PLEG): container finished" podID="6a655f0e-b533-471c-b036-e45ff86bc2d3" containerID="a8f5e4ecf8f71e5df9c5ec6e96b9e0bb9d240a17dcf48649cfea1728bca52a69" exitCode=0 Jul 21 13:51:50.943139 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:50.943092 2567 generic.go:356] "Generic (PLEG): container finished" podID="570dd1e5-86c8-4877-9e8b-ed536ce227df" containerID="348bc59be7050f21a1cefc476fa94c93f55be30646bb423373c966774203d767" exitCode=0 Jul 21 13:51:51.004325 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.004285 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r77z\" (UniqueName: \"kubernetes.io/projected/5f4b4477-39a1-4f98-9d08-d70bcffb489a-kube-api-access-9r77z\") pod \"limitador-operator-controller-manager-859d7b78bd-jpbms\" (UID: \"5f4b4477-39a1-4f98-9d08-d70bcffb489a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jpbms" Jul 21 13:51:51.018286 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.018256 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r77z\" (UniqueName: \"kubernetes.io/projected/5f4b4477-39a1-4f98-9d08-d70bcffb489a-kube-api-access-9r77z\") pod \"limitador-operator-controller-manager-859d7b78bd-jpbms\" (UID: \"5f4b4477-39a1-4f98-9d08-d70bcffb489a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jpbms" Jul 21 13:51:51.022008 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.021964 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" Jul 21 13:51:51.042096 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.041999 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:51.044677 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:51:51.044626 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnzmf\" is forbidden: User \"system:node:ip-10-0-129-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-212.ec2.internal' and this object" podUID="570dd1e5-86c8-4877-9e8b-ed536ce227df" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" Jul 21 13:51:51.047047 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:51:51.047018 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7pnv\" is forbidden: User \"system:node:ip-10-0-129-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-212.ec2.internal' and this object" podUID="6a655f0e-b533-471c-b036-e45ff86bc2d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:51.050384 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.050362 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" Jul 21 13:51:51.053048 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:51:51.053022 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qnzmf\" is forbidden: User \"system:node:ip-10-0-129-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-212.ec2.internal' and this object" podUID="570dd1e5-86c8-4877-9e8b-ed536ce227df" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" Jul 21 13:51:51.055311 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:51:51.055282 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l7pnv\" is forbidden: User \"system:node:ip-10-0-129-212.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-212.ec2.internal' and this object" podUID="6a655f0e-b533-471c-b036-e45ff86bc2d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:51.105652 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.105620 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v5t7\" (UniqueName: \"kubernetes.io/projected/6a655f0e-b533-471c-b036-e45ff86bc2d3-kube-api-access-6v5t7\") pod \"6a655f0e-b533-471c-b036-e45ff86bc2d3\" (UID: \"6a655f0e-b533-471c-b036-e45ff86bc2d3\") " Jul 21 13:51:51.105976 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.105693 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzrlc\" (UniqueName: \"kubernetes.io/projected/570dd1e5-86c8-4877-9e8b-ed536ce227df-kube-api-access-xzrlc\") pod \"570dd1e5-86c8-4877-9e8b-ed536ce227df\" (UID: \"570dd1e5-86c8-4877-9e8b-ed536ce227df\") " Jul 21 13:51:51.105976 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.105737 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a655f0e-b533-471c-b036-e45ff86bc2d3-extensions-socket-volume\") pod \"6a655f0e-b533-471c-b036-e45ff86bc2d3\" (UID: \"6a655f0e-b533-471c-b036-e45ff86bc2d3\") " Jul 21 13:51:51.106282 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.106251 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a655f0e-b533-471c-b036-e45ff86bc2d3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6a655f0e-b533-471c-b036-e45ff86bc2d3" (UID: "6a655f0e-b533-471c-b036-e45ff86bc2d3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:51:51.107756 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.107728 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/570dd1e5-86c8-4877-9e8b-ed536ce227df-kube-api-access-xzrlc" (OuterVolumeSpecName: "kube-api-access-xzrlc") pod "570dd1e5-86c8-4877-9e8b-ed536ce227df" (UID: "570dd1e5-86c8-4877-9e8b-ed536ce227df"). InnerVolumeSpecName "kube-api-access-xzrlc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:51:51.108063 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.108038 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a655f0e-b533-471c-b036-e45ff86bc2d3-kube-api-access-6v5t7" (OuterVolumeSpecName: "kube-api-access-6v5t7") pod "6a655f0e-b533-471c-b036-e45ff86bc2d3" (UID: "6a655f0e-b533-471c-b036-e45ff86bc2d3"). InnerVolumeSpecName "kube-api-access-6v5t7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:51:51.172551 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.172523 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6"] Jul 21 13:51:51.175129 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:51:51.175100 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1848a5d0_54ec_4255_a1ef_4a2f0a7bd059.slice/crio-93f492028ab3e64782043ac2c1debddfffe6ee3ff12b0b3b30bac13792569504 WatchSource:0}: Error finding container 93f492028ab3e64782043ac2c1debddfffe6ee3ff12b0b3b30bac13792569504: Status 404 returned error can't find the container with id 93f492028ab3e64782043ac2c1debddfffe6ee3ff12b0b3b30bac13792569504 Jul 21 13:51:51.206692 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.206662 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzrlc\" (UniqueName: \"kubernetes.io/projected/570dd1e5-86c8-4877-9e8b-ed536ce227df-kube-api-access-xzrlc\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:51.206692 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.206694 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a655f0e-b533-471c-b036-e45ff86bc2d3-extensions-socket-volume\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:51.206878 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.206724 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v5t7\" (UniqueName: \"kubernetes.io/projected/6a655f0e-b533-471c-b036-e45ff86bc2d3-kube-api-access-6v5t7\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:51:51.216538 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.216497 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jpbms" Jul 21 13:51:51.360605 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.360578 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jpbms"] Jul 21 13:51:51.362262 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:51:51.362241 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f4b4477_39a1_4f98_9d08_d70bcffb489a.slice/crio-ce3b7df20473f275109028e4ea56d1d69bc531b9e8c69a0cf7307bfbd773beb6 WatchSource:0}: Error finding container ce3b7df20473f275109028e4ea56d1d69bc531b9e8c69a0cf7307bfbd773beb6: Status 404 returned error can't find the container with id ce3b7df20473f275109028e4ea56d1d69bc531b9e8c69a0cf7307bfbd773beb6 Jul 21 13:51:51.533581 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.533487 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="570dd1e5-86c8-4877-9e8b-ed536ce227df" path="/var/lib/kubelet/pods/570dd1e5-86c8-4877-9e8b-ed536ce227df/volumes" Jul 21 13:51:51.533975 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.533956 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a655f0e-b533-471c-b036-e45ff86bc2d3" path="/var/lib/kubelet/pods/6a655f0e-b533-471c-b036-e45ff86bc2d3/volumes" Jul 21 13:51:51.949650 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.949619 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l7pnv" Jul 21 13:51:51.950088 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.949673 2567 scope.go:117] "RemoveContainer" containerID="a8f5e4ecf8f71e5df9c5ec6e96b9e0bb9d240a17dcf48649cfea1728bca52a69" Jul 21 13:51:51.951163 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.951144 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnzmf" Jul 21 13:51:51.953146 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.953110 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" event={"ID":"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059","Type":"ContainerStarted","Data":"437e92245fe2fcacc84ba700335464ce3138d65f6b53f345bfb703f807b11d91"} Jul 21 13:51:51.953260 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.953172 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" Jul 21 13:51:51.953260 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.953199 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" event={"ID":"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059","Type":"ContainerStarted","Data":"93f492028ab3e64782043ac2c1debddfffe6ee3ff12b0b3b30bac13792569504"} Jul 21 13:51:51.955029 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.955000 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jpbms" event={"ID":"5f4b4477-39a1-4f98-9d08-d70bcffb489a","Type":"ContainerStarted","Data":"c6dd2e50a68b3e5624f0c8ab3afcbc21792c3ecad62ad2d02e32e0053bf89c0f"} Jul 21 13:51:51.955125 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.955054 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jpbms" event={"ID":"5f4b4477-39a1-4f98-9d08-d70bcffb489a","Type":"ContainerStarted","Data":"ce3b7df20473f275109028e4ea56d1d69bc531b9e8c69a0cf7307bfbd773beb6"} Jul 21 13:51:51.955125 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.955106 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jpbms" Jul 21 13:51:51.959378 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.959360 2567 scope.go:117] "RemoveContainer" containerID="348bc59be7050f21a1cefc476fa94c93f55be30646bb423373c966774203d767" Jul 21 13:51:51.991271 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:51.991214 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" podStartSLOduration=1.991198168 podStartE2EDuration="1.991198168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:51:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:51:51.98756724 +0000 UTC m=+473.139408350" watchObservedRunningTime="2026-07-21 13:51:51.991198168 +0000 UTC m=+473.143039329" Jul 21 13:51:52.039024 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:52.038971 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jpbms" podStartSLOduration=2.038952725 podStartE2EDuration="2.038952725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:51:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:51:52.038395279 +0000 UTC m=+473.190236401" watchObservedRunningTime="2026-07-21 13:51:52.038952725 +0000 UTC m=+473.190793902" Jul 21 13:51:52.500127 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:52.500089 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:52.616638 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:52.616596 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:55.682166 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:55.682124 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:55.770205 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:55.770163 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:55.841866 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:55.841834 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:55.943650 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:55.943558 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:59.762486 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:59.762448 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:51:59.940096 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:51:59.940064 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:02.965870 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:02.965829 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jpbms" Jul 21 13:52:02.966235 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:02.966029 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" Jul 21 13:52:05.285836 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:05.285795 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:05.427753 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:05.427713 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:17.157351 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:17.157312 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:17.220837 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:17.220800 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:19.552110 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.552074 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x"] Jul 21 13:52:19.555601 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.555578 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.558847 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.558827 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-5j9v6\"" Jul 21 13:52:19.567401 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.567374 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x"] Jul 21 13:52:19.662647 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.662608 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a886230-716b-4ef6-97ae-0847508d4ab7-istio-token\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.662647 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.662649 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-workload-socket\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.662894 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.662675 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-workload-certs\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.662894 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.662788 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a886230-716b-4ef6-97ae-0847508d4ab7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.662894 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.662822 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-istio-envoy\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.662894 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.662843 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a886230-716b-4ef6-97ae-0847508d4ab7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.663104 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.662917 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-istio-data\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.663104 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.662944 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-credential-socket\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.663104 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.662975 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtmn5\" (UniqueName: \"kubernetes.io/projected/5a886230-716b-4ef6-97ae-0847508d4ab7-kube-api-access-vtmn5\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.671951 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.671923 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:19.763580 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.763537 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtmn5\" (UniqueName: \"kubernetes.io/projected/5a886230-716b-4ef6-97ae-0847508d4ab7-kube-api-access-vtmn5\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.763817 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.763601 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a886230-716b-4ef6-97ae-0847508d4ab7-istio-token\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.763817 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.763637 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-workload-socket\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.763817 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.763672 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-workload-certs\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.763817 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.763748 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a886230-716b-4ef6-97ae-0847508d4ab7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.763817 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.763779 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-istio-envoy\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.763817 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.763806 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a886230-716b-4ef6-97ae-0847508d4ab7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.764142 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.763861 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-istio-data\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.764142 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.763894 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-credential-socket\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.764269 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.764172 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-workload-socket\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.764269 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.764219 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-credential-socket\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.764269 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.764252 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-istio-data\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.764435 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.764373 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-workload-certs\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.764750 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.764726 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a886230-716b-4ef6-97ae-0847508d4ab7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.766363 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.766343 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a886230-716b-4ef6-97ae-0847508d4ab7-istio-envoy\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.766421 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.766372 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a886230-716b-4ef6-97ae-0847508d4ab7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.772346 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.772322 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a886230-716b-4ef6-97ae-0847508d4ab7-istio-token\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.772684 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.772642 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtmn5\" (UniqueName: \"kubernetes.io/projected/5a886230-716b-4ef6-97ae-0847508d4ab7-kube-api-access-vtmn5\") pod \"maas-default-gateway-openshift-default-74949dcd89-dlm5x\" (UID: \"5a886230-716b-4ef6-97ae-0847508d4ab7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:19.774319 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.774274 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:19.866871 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:19.866774 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:20.017799 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:20.017766 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x"] Jul 21 13:52:20.019664 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:52:20.019639 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a886230_716b_4ef6_97ae_0847508d4ab7.slice/crio-0a74c682a45f15bead33d9251c3be12dd9817272e9110f7a4f3d3f63b08fc2a9 WatchSource:0}: Error finding container 0a74c682a45f15bead33d9251c3be12dd9817272e9110f7a4f3d3f63b08fc2a9: Status 404 returned error can't find the container with id 0a74c682a45f15bead33d9251c3be12dd9817272e9110f7a4f3d3f63b08fc2a9 Jul 21 13:52:20.022039 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:20.022006 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 13:52:20.022109 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:20.022077 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 13:52:20.022149 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:20.022110 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 13:52:20.114096 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:20.114061 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" event={"ID":"5a886230-716b-4ef6-97ae-0847508d4ab7","Type":"ContainerStarted","Data":"f66f86bb71548d9c7ccb18401186c4c9da786ed67e939a22c3b3a35b5280c530"} Jul 21 13:52:20.114254 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:20.114131 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" event={"ID":"5a886230-716b-4ef6-97ae-0847508d4ab7","Type":"ContainerStarted","Data":"0a74c682a45f15bead33d9251c3be12dd9817272e9110f7a4f3d3f63b08fc2a9"} Jul 21 13:52:20.142631 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:20.142508 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" podStartSLOduration=1.142487252 podStartE2EDuration="1.142487252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:52:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:52:20.141754225 +0000 UTC m=+501.293595335" watchObservedRunningTime="2026-07-21 13:52:20.142487252 +0000 UTC m=+501.294328364" Jul 21 13:52:20.867822 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:20.867775 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:21.314766 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:21.314720 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:21.396283 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:21.396244 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:21.873031 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:21.872998 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:22.124556 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.124472 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:22.125512 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.125494 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:52:22.167758 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.167720 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-xsmb6"] Jul 21 13:52:22.171236 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.171217 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xsmb6" Jul 21 13:52:22.175622 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.175598 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xrwj6\"" Jul 21 13:52:22.184429 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.184402 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xsmb6"] Jul 21 13:52:22.225510 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.225476 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:22.289008 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.288963 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9r6m\" (UniqueName: \"kubernetes.io/projected/30570325-cdb0-4631-a235-fd9e0dd4d514-kube-api-access-w9r6m\") pod \"authorino-58cd955675-xsmb6\" (UID: \"30570325-cdb0-4631-a235-fd9e0dd4d514\") " pod="kuadrant-system/authorino-58cd955675-xsmb6" Jul 21 13:52:22.389788 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.389654 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9r6m\" (UniqueName: \"kubernetes.io/projected/30570325-cdb0-4631-a235-fd9e0dd4d514-kube-api-access-w9r6m\") pod \"authorino-58cd955675-xsmb6\" (UID: \"30570325-cdb0-4631-a235-fd9e0dd4d514\") " pod="kuadrant-system/authorino-58cd955675-xsmb6" Jul 21 13:52:22.400631 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.400598 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9r6m\" (UniqueName: \"kubernetes.io/projected/30570325-cdb0-4631-a235-fd9e0dd4d514-kube-api-access-w9r6m\") pod \"authorino-58cd955675-xsmb6\" (UID: \"30570325-cdb0-4631-a235-fd9e0dd4d514\") " pod="kuadrant-system/authorino-58cd955675-xsmb6" Jul 21 13:52:22.483581 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.483535 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xsmb6" Jul 21 13:52:22.495237 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.495209 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmvnc"] Jul 21 13:52:22.498770 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.498743 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mmvnc" Jul 21 13:52:22.511483 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.511453 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmvnc"] Jul 21 13:52:22.541829 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.541788 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:22.592314 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.592274 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjh2c\" (UniqueName: \"kubernetes.io/projected/e2c0c579-6654-42c3-8164-f9af35a49535-kube-api-access-fjh2c\") pod \"authorino-75d556bd77-mmvnc\" (UID: \"e2c0c579-6654-42c3-8164-f9af35a49535\") " pod="kuadrant-system/authorino-75d556bd77-mmvnc" Jul 21 13:52:22.634862 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.634769 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xsmb6"] Jul 21 13:52:22.637111 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:52:22.637076 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30570325_cdb0_4631_a235_fd9e0dd4d514.slice/crio-cb31ade0f3792754cf92f88e600f46e160d84406586372e0596cf4cb161b3260 WatchSource:0}: Error finding container cb31ade0f3792754cf92f88e600f46e160d84406586372e0596cf4cb161b3260: Status 404 returned error can't find the container with id cb31ade0f3792754cf92f88e600f46e160d84406586372e0596cf4cb161b3260 Jul 21 13:52:22.693138 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.693090 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjh2c\" (UniqueName: \"kubernetes.io/projected/e2c0c579-6654-42c3-8164-f9af35a49535-kube-api-access-fjh2c\") pod \"authorino-75d556bd77-mmvnc\" (UID: \"e2c0c579-6654-42c3-8164-f9af35a49535\") " pod="kuadrant-system/authorino-75d556bd77-mmvnc" Jul 21 13:52:22.702127 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.702099 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjh2c\" (UniqueName: \"kubernetes.io/projected/e2c0c579-6654-42c3-8164-f9af35a49535-kube-api-access-fjh2c\") pod \"authorino-75d556bd77-mmvnc\" (UID: \"e2c0c579-6654-42c3-8164-f9af35a49535\") " pod="kuadrant-system/authorino-75d556bd77-mmvnc" Jul 21 13:52:22.818248 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.818200 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mmvnc" Jul 21 13:52:22.992078 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:22.992009 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmvnc"] Jul 21 13:52:22.994129 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:52:22.994096 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2c0c579_6654_42c3_8164_f9af35a49535.slice/crio-33eaac487b2a7255e3a04d7305b8abca95c6c8ff4b9ebed53feff20d1b5e22b1 WatchSource:0}: Error finding container 33eaac487b2a7255e3a04d7305b8abca95c6c8ff4b9ebed53feff20d1b5e22b1: Status 404 returned error can't find the container with id 33eaac487b2a7255e3a04d7305b8abca95c6c8ff4b9ebed53feff20d1b5e22b1 Jul 21 13:52:23.130434 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:23.130401 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mmvnc" event={"ID":"e2c0c579-6654-42c3-8164-f9af35a49535","Type":"ContainerStarted","Data":"33eaac487b2a7255e3a04d7305b8abca95c6c8ff4b9ebed53feff20d1b5e22b1"} Jul 21 13:52:23.131592 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:23.131563 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xsmb6" event={"ID":"30570325-cdb0-4631-a235-fd9e0dd4d514","Type":"ContainerStarted","Data":"cb31ade0f3792754cf92f88e600f46e160d84406586372e0596cf4cb161b3260"} Jul 21 13:52:27.161992 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:27.161955 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xsmb6" event={"ID":"30570325-cdb0-4631-a235-fd9e0dd4d514","Type":"ContainerStarted","Data":"2a3476bd6d40017f5dc1375dde1e562f9acb77756680b3502a25ecabc4d32986"} Jul 21 13:52:27.163499 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:27.163476 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mmvnc" event={"ID":"e2c0c579-6654-42c3-8164-f9af35a49535","Type":"ContainerStarted","Data":"8ac721da5035964aa1d8dfd68c1a4243c08359a214fdc59e0ce948e3f7da0e95"} Jul 21 13:52:27.185434 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:27.185379 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-xsmb6" podStartSLOduration=1.3993425689999999 podStartE2EDuration="5.185364688s" totalImagesPullingTime="3.786022119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:52:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:52:27.182558672 +0000 UTC m=+508.334399778" watchObservedRunningTime="2026-07-21 13:52:27.185364688 +0000 UTC m=+508.337205797" Jul 21 13:52:27.208581 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:27.208508 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-mmvnc" podStartSLOduration=1.762475504 podStartE2EDuration="5.208480126s" totalImagesPullingTime="3.446004622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:52:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:52:27.205477745 +0000 UTC m=+508.357318854" watchObservedRunningTime="2026-07-21 13:52:27.208480126 +0000 UTC m=+508.360321236" Jul 21 13:52:27.234628 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:27.234590 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xsmb6"] Jul 21 13:52:29.173714 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:29.173665 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-xsmb6" podUID="30570325-cdb0-4631-a235-fd9e0dd4d514" containerName="authorino" containerID="cri-o://2a3476bd6d40017f5dc1375dde1e562f9acb77756680b3502a25ecabc4d32986" gracePeriod=30 Jul 21 13:52:29.427523 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:29.427457 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xsmb6" Jul 21 13:52:29.558366 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:29.558320 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9r6m\" (UniqueName: \"kubernetes.io/projected/30570325-cdb0-4631-a235-fd9e0dd4d514-kube-api-access-w9r6m\") pod \"30570325-cdb0-4631-a235-fd9e0dd4d514\" (UID: \"30570325-cdb0-4631-a235-fd9e0dd4d514\") " Jul 21 13:52:29.560482 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:29.560446 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30570325-cdb0-4631-a235-fd9e0dd4d514-kube-api-access-w9r6m" (OuterVolumeSpecName: "kube-api-access-w9r6m") pod "30570325-cdb0-4631-a235-fd9e0dd4d514" (UID: "30570325-cdb0-4631-a235-fd9e0dd4d514"). InnerVolumeSpecName "kube-api-access-w9r6m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:52:29.659148 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:29.659104 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9r6m\" (UniqueName: \"kubernetes.io/projected/30570325-cdb0-4631-a235-fd9e0dd4d514-kube-api-access-w9r6m\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:52:30.180457 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:30.180418 2567 generic.go:356] "Generic (PLEG): container finished" podID="30570325-cdb0-4631-a235-fd9e0dd4d514" containerID="2a3476bd6d40017f5dc1375dde1e562f9acb77756680b3502a25ecabc4d32986" exitCode=0 Jul 21 13:52:30.180919 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:30.180476 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xsmb6" Jul 21 13:52:30.180919 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:30.180505 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xsmb6" event={"ID":"30570325-cdb0-4631-a235-fd9e0dd4d514","Type":"ContainerDied","Data":"2a3476bd6d40017f5dc1375dde1e562f9acb77756680b3502a25ecabc4d32986"} Jul 21 13:52:30.180919 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:30.180553 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xsmb6" event={"ID":"30570325-cdb0-4631-a235-fd9e0dd4d514","Type":"ContainerDied","Data":"cb31ade0f3792754cf92f88e600f46e160d84406586372e0596cf4cb161b3260"} Jul 21 13:52:30.180919 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:30.180572 2567 scope.go:117] "RemoveContainer" containerID="2a3476bd6d40017f5dc1375dde1e562f9acb77756680b3502a25ecabc4d32986" Jul 21 13:52:30.190819 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:30.190789 2567 scope.go:117] "RemoveContainer" containerID="2a3476bd6d40017f5dc1375dde1e562f9acb77756680b3502a25ecabc4d32986" Jul 21 13:52:30.191117 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:52:30.191098 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a3476bd6d40017f5dc1375dde1e562f9acb77756680b3502a25ecabc4d32986\": container with ID starting with 2a3476bd6d40017f5dc1375dde1e562f9acb77756680b3502a25ecabc4d32986 not found: ID does not exist" containerID="2a3476bd6d40017f5dc1375dde1e562f9acb77756680b3502a25ecabc4d32986" Jul 21 13:52:30.191163 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:30.191127 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3476bd6d40017f5dc1375dde1e562f9acb77756680b3502a25ecabc4d32986"} err="failed to get container status \"2a3476bd6d40017f5dc1375dde1e562f9acb77756680b3502a25ecabc4d32986\": rpc error: code = NotFound desc = could not find container \"2a3476bd6d40017f5dc1375dde1e562f9acb77756680b3502a25ecabc4d32986\": container with ID starting with 2a3476bd6d40017f5dc1375dde1e562f9acb77756680b3502a25ecabc4d32986 not found: ID does not exist" Jul 21 13:52:30.205506 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:30.205457 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xsmb6"] Jul 21 13:52:30.207851 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:30.207830 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-xsmb6"] Jul 21 13:52:31.533109 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:31.533071 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30570325-cdb0-4631-a235-fd9e0dd4d514" path="/var/lib/kubelet/pods/30570325-cdb0-4631-a235-fd9e0dd4d514/volumes" Jul 21 13:52:42.485278 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:42.485232 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:42.591232 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:42.591192 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:42.859412 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:42.859300 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:52.405132 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:52.405046 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:52.507378 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:52.507331 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:54.643792 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:54.643739 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:54.850342 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:54.850303 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vjx94"] Jul 21 13:52:54.850781 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:54.850761 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30570325-cdb0-4631-a235-fd9e0dd4d514" containerName="authorino" Jul 21 13:52:54.850781 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:54.850781 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="30570325-cdb0-4631-a235-fd9e0dd4d514" containerName="authorino" Jul 21 13:52:54.850900 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:54.850858 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="30570325-cdb0-4631-a235-fd9e0dd4d514" containerName="authorino" Jul 21 13:52:54.890588 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:54.890548 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vjx94"] Jul 21 13:52:54.890763 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:54.890668 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vjx94" Jul 21 13:52:54.989949 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:54.989912 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28pmv\" (UniqueName: \"kubernetes.io/projected/05cbe06d-4169-4acc-be5c-920863b2114d-kube-api-access-28pmv\") pod \"authorino-5d4b67c446-vjx94\" (UID: \"05cbe06d-4169-4acc-be5c-920863b2114d\") " pod="kuadrant-system/authorino-5d4b67c446-vjx94" Jul 21 13:52:55.090628 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.090587 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28pmv\" (UniqueName: \"kubernetes.io/projected/05cbe06d-4169-4acc-be5c-920863b2114d-kube-api-access-28pmv\") pod \"authorino-5d4b67c446-vjx94\" (UID: \"05cbe06d-4169-4acc-be5c-920863b2114d\") " pod="kuadrant-system/authorino-5d4b67c446-vjx94" Jul 21 13:52:55.093551 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.093519 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vjx94"] Jul 21 13:52:55.093812 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:52:55.093788 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-28pmv], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-vjx94" podUID="05cbe06d-4169-4acc-be5c-920863b2114d" Jul 21 13:52:55.102044 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.102014 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28pmv\" (UniqueName: \"kubernetes.io/projected/05cbe06d-4169-4acc-be5c-920863b2114d-kube-api-access-28pmv\") pod \"authorino-5d4b67c446-vjx94\" (UID: \"05cbe06d-4169-4acc-be5c-920863b2114d\") " pod="kuadrant-system/authorino-5d4b67c446-vjx94" Jul 21 13:52:55.110453 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.110423 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:52:55.128212 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.128172 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67b8dc6c88-bhr6w"] Jul 21 13:52:55.131772 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.131744 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67b8dc6c88-bhr6w" Jul 21 13:52:55.141375 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.141345 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67b8dc6c88-bhr6w"] Jul 21 13:52:55.191480 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.191435 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bzmn\" (UniqueName: \"kubernetes.io/projected/d26f7052-d54f-4e0e-8063-4571da7db18f-kube-api-access-8bzmn\") pod \"authorino-67b8dc6c88-bhr6w\" (UID: \"d26f7052-d54f-4e0e-8063-4571da7db18f\") " pod="kuadrant-system/authorino-67b8dc6c88-bhr6w" Jul 21 13:52:55.292639 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.292594 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bzmn\" (UniqueName: \"kubernetes.io/projected/d26f7052-d54f-4e0e-8063-4571da7db18f-kube-api-access-8bzmn\") pod \"authorino-67b8dc6c88-bhr6w\" (UID: \"d26f7052-d54f-4e0e-8063-4571da7db18f\") " pod="kuadrant-system/authorino-67b8dc6c88-bhr6w" Jul 21 13:52:55.304194 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.304164 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bzmn\" (UniqueName: \"kubernetes.io/projected/d26f7052-d54f-4e0e-8063-4571da7db18f-kube-api-access-8bzmn\") pod \"authorino-67b8dc6c88-bhr6w\" (UID: \"d26f7052-d54f-4e0e-8063-4571da7db18f\") " pod="kuadrant-system/authorino-67b8dc6c88-bhr6w" Jul 21 13:52:55.313441 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.313413 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vjx94" Jul 21 13:52:55.319638 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.319609 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vjx94" Jul 21 13:52:55.327186 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.327160 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67b8dc6c88-bhr6w"] Jul 21 13:52:55.327435 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.327422 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67b8dc6c88-bhr6w" Jul 21 13:52:55.361759 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.361721 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9978846dd-qc989"] Jul 21 13:52:55.365113 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.365085 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9978846dd-qc989" Jul 21 13:52:55.368372 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.368338 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 21 13:52:55.374803 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.374773 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9978846dd-qc989"] Jul 21 13:52:55.393172 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.393097 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28pmv\" (UniqueName: \"kubernetes.io/projected/05cbe06d-4169-4acc-be5c-920863b2114d-kube-api-access-28pmv\") pod \"05cbe06d-4169-4acc-be5c-920863b2114d\" (UID: \"05cbe06d-4169-4acc-be5c-920863b2114d\") " Jul 21 13:52:55.397993 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.397928 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05cbe06d-4169-4acc-be5c-920863b2114d-kube-api-access-28pmv" (OuterVolumeSpecName: "kube-api-access-28pmv") pod "05cbe06d-4169-4acc-be5c-920863b2114d" (UID: "05cbe06d-4169-4acc-be5c-920863b2114d"). InnerVolumeSpecName "kube-api-access-28pmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:52:55.474146 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.474119 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67b8dc6c88-bhr6w"] Jul 21 13:52:55.475929 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:52:55.475890 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd26f7052_d54f_4e0e_8063_4571da7db18f.slice/crio-adc3eeadf99ed55fb477f0e64d87d0187ef5c35c2ed79ef0daf564fc9e2ec06e WatchSource:0}: Error finding container adc3eeadf99ed55fb477f0e64d87d0187ef5c35c2ed79ef0daf564fc9e2ec06e: Status 404 returned error can't find the container with id adc3eeadf99ed55fb477f0e64d87d0187ef5c35c2ed79ef0daf564fc9e2ec06e Jul 21 13:52:55.494043 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.494013 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba660e25-8f20-49cb-9675-26a628a75067-tls-cert\") pod \"authorino-9978846dd-qc989\" (UID: \"ba660e25-8f20-49cb-9675-26a628a75067\") " pod="kuadrant-system/authorino-9978846dd-qc989" Jul 21 13:52:55.494178 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.494083 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w97sg\" (UniqueName: \"kubernetes.io/projected/ba660e25-8f20-49cb-9675-26a628a75067-kube-api-access-w97sg\") pod \"authorino-9978846dd-qc989\" (UID: \"ba660e25-8f20-49cb-9675-26a628a75067\") " pod="kuadrant-system/authorino-9978846dd-qc989" Jul 21 13:52:55.494219 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.494194 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28pmv\" (UniqueName: \"kubernetes.io/projected/05cbe06d-4169-4acc-be5c-920863b2114d-kube-api-access-28pmv\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:52:55.595494 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.595451 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w97sg\" (UniqueName: \"kubernetes.io/projected/ba660e25-8f20-49cb-9675-26a628a75067-kube-api-access-w97sg\") pod \"authorino-9978846dd-qc989\" (UID: \"ba660e25-8f20-49cb-9675-26a628a75067\") " pod="kuadrant-system/authorino-9978846dd-qc989" Jul 21 13:52:55.595806 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.595771 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba660e25-8f20-49cb-9675-26a628a75067-tls-cert\") pod \"authorino-9978846dd-qc989\" (UID: \"ba660e25-8f20-49cb-9675-26a628a75067\") " pod="kuadrant-system/authorino-9978846dd-qc989" Jul 21 13:52:55.599733 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.599685 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba660e25-8f20-49cb-9675-26a628a75067-tls-cert\") pod \"authorino-9978846dd-qc989\" (UID: \"ba660e25-8f20-49cb-9675-26a628a75067\") " pod="kuadrant-system/authorino-9978846dd-qc989" Jul 21 13:52:55.605042 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.605016 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w97sg\" (UniqueName: \"kubernetes.io/projected/ba660e25-8f20-49cb-9675-26a628a75067-kube-api-access-w97sg\") pod \"authorino-9978846dd-qc989\" (UID: \"ba660e25-8f20-49cb-9675-26a628a75067\") " pod="kuadrant-system/authorino-9978846dd-qc989" Jul 21 13:52:55.678717 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.678650 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9978846dd-qc989" Jul 21 13:52:55.817073 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:55.817039 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9978846dd-qc989"] Jul 21 13:52:55.818863 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:52:55.818833 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba660e25_8f20_49cb_9675_26a628a75067.slice/crio-87ac0a0759c67f7222e0817418ea31ee0221d6bdc28719da4cee9921edc26c47 WatchSource:0}: Error finding container 87ac0a0759c67f7222e0817418ea31ee0221d6bdc28719da4cee9921edc26c47: Status 404 returned error can't find the container with id 87ac0a0759c67f7222e0817418ea31ee0221d6bdc28719da4cee9921edc26c47 Jul 21 13:52:56.321052 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:56.321019 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9978846dd-qc989" event={"ID":"ba660e25-8f20-49cb-9675-26a628a75067","Type":"ContainerStarted","Data":"87ac0a0759c67f7222e0817418ea31ee0221d6bdc28719da4cee9921edc26c47"} Jul 21 13:52:56.322820 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:56.322733 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67b8dc6c88-bhr6w" event={"ID":"d26f7052-d54f-4e0e-8063-4571da7db18f","Type":"ContainerStarted","Data":"7198ac690a5e1c3ad368fb46db9eb28da77a3d016ad9cbcfba662afe92eb5024"} Jul 21 13:52:56.322820 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:56.322778 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-67b8dc6c88-bhr6w" podUID="d26f7052-d54f-4e0e-8063-4571da7db18f" containerName="authorino" containerID="cri-o://7198ac690a5e1c3ad368fb46db9eb28da77a3d016ad9cbcfba662afe92eb5024" gracePeriod=30 Jul 21 13:52:56.322820 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:56.322790 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67b8dc6c88-bhr6w" event={"ID":"d26f7052-d54f-4e0e-8063-4571da7db18f","Type":"ContainerStarted","Data":"adc3eeadf99ed55fb477f0e64d87d0187ef5c35c2ed79ef0daf564fc9e2ec06e"} Jul 21 13:52:56.322820 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:56.322793 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vjx94" Jul 21 13:52:56.344566 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:56.344519 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-67b8dc6c88-bhr6w" podStartSLOduration=0.977484731 podStartE2EDuration="1.344502717s" totalImagesPullingTime="367.017986ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:52:56.34277137 +0000 UTC m=+537.494612479" watchObservedRunningTime="2026-07-21 13:52:56.344502717 +0000 UTC m=+537.496343825" Jul 21 13:52:56.410157 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:56.410111 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vjx94"] Jul 21 13:52:56.414999 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:56.414970 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vjx94"] Jul 21 13:52:56.577505 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:56.577477 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67b8dc6c88-bhr6w" Jul 21 13:52:56.705408 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:56.705370 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bzmn\" (UniqueName: \"kubernetes.io/projected/d26f7052-d54f-4e0e-8063-4571da7db18f-kube-api-access-8bzmn\") pod \"d26f7052-d54f-4e0e-8063-4571da7db18f\" (UID: \"d26f7052-d54f-4e0e-8063-4571da7db18f\") " Jul 21 13:52:56.707537 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:56.707509 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d26f7052-d54f-4e0e-8063-4571da7db18f-kube-api-access-8bzmn" (OuterVolumeSpecName: "kube-api-access-8bzmn") pod "d26f7052-d54f-4e0e-8063-4571da7db18f" (UID: "d26f7052-d54f-4e0e-8063-4571da7db18f"). InnerVolumeSpecName "kube-api-access-8bzmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:52:56.806909 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:56.806868 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bzmn\" (UniqueName: \"kubernetes.io/projected/d26f7052-d54f-4e0e-8063-4571da7db18f-kube-api-access-8bzmn\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:52:57.330723 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.330667 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9978846dd-qc989" event={"ID":"ba660e25-8f20-49cb-9675-26a628a75067","Type":"ContainerStarted","Data":"61187c881fca7afaa62ba739fec664f00bef34a77335c273d736fc0a1ff2a083"} Jul 21 13:52:57.331887 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.331858 2567 generic.go:356] "Generic (PLEG): container finished" podID="d26f7052-d54f-4e0e-8063-4571da7db18f" containerID="7198ac690a5e1c3ad368fb46db9eb28da77a3d016ad9cbcfba662afe92eb5024" exitCode=0 Jul 21 13:52:57.332011 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.331891 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67b8dc6c88-bhr6w" Jul 21 13:52:57.332011 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.331920 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67b8dc6c88-bhr6w" event={"ID":"d26f7052-d54f-4e0e-8063-4571da7db18f","Type":"ContainerDied","Data":"7198ac690a5e1c3ad368fb46db9eb28da77a3d016ad9cbcfba662afe92eb5024"} Jul 21 13:52:57.332011 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.331952 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67b8dc6c88-bhr6w" event={"ID":"d26f7052-d54f-4e0e-8063-4571da7db18f","Type":"ContainerDied","Data":"adc3eeadf99ed55fb477f0e64d87d0187ef5c35c2ed79ef0daf564fc9e2ec06e"} Jul 21 13:52:57.332011 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.331971 2567 scope.go:117] "RemoveContainer" containerID="7198ac690a5e1c3ad368fb46db9eb28da77a3d016ad9cbcfba662afe92eb5024" Jul 21 13:52:57.341768 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.341746 2567 scope.go:117] "RemoveContainer" containerID="7198ac690a5e1c3ad368fb46db9eb28da77a3d016ad9cbcfba662afe92eb5024" Jul 21 13:52:57.342107 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:52:57.342087 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7198ac690a5e1c3ad368fb46db9eb28da77a3d016ad9cbcfba662afe92eb5024\": container with ID starting with 7198ac690a5e1c3ad368fb46db9eb28da77a3d016ad9cbcfba662afe92eb5024 not found: ID does not exist" containerID="7198ac690a5e1c3ad368fb46db9eb28da77a3d016ad9cbcfba662afe92eb5024" Jul 21 13:52:57.342152 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.342117 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7198ac690a5e1c3ad368fb46db9eb28da77a3d016ad9cbcfba662afe92eb5024"} err="failed to get container status \"7198ac690a5e1c3ad368fb46db9eb28da77a3d016ad9cbcfba662afe92eb5024\": rpc error: code = NotFound desc = could not find container \"7198ac690a5e1c3ad368fb46db9eb28da77a3d016ad9cbcfba662afe92eb5024\": container with ID starting with 7198ac690a5e1c3ad368fb46db9eb28da77a3d016ad9cbcfba662afe92eb5024 not found: ID does not exist" Jul 21 13:52:57.364442 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.364341 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-9978846dd-qc989" podStartSLOduration=1.9081773850000001 podStartE2EDuration="2.3643257s" totalImagesPullingTime="456.148315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:52:57.358293761 +0000 UTC m=+538.510134870" watchObservedRunningTime="2026-07-21 13:52:57.3643257 +0000 UTC m=+538.516166809" Jul 21 13:52:57.390121 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.390085 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67b8dc6c88-bhr6w"] Jul 21 13:52:57.398570 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.398533 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-67b8dc6c88-bhr6w"] Jul 21 13:52:57.411422 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.411386 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmvnc"] Jul 21 13:52:57.411625 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.411600 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-mmvnc" podUID="e2c0c579-6654-42c3-8164-f9af35a49535" containerName="authorino" containerID="cri-o://8ac721da5035964aa1d8dfd68c1a4243c08359a214fdc59e0ce948e3f7da0e95" gracePeriod=30 Jul 21 13:52:57.533663 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.533619 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05cbe06d-4169-4acc-be5c-920863b2114d" path="/var/lib/kubelet/pods/05cbe06d-4169-4acc-be5c-920863b2114d/volumes" Jul 21 13:52:57.533898 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.533882 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d26f7052-d54f-4e0e-8063-4571da7db18f" path="/var/lib/kubelet/pods/d26f7052-d54f-4e0e-8063-4571da7db18f/volumes" Jul 21 13:52:57.675943 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.675909 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mmvnc" Jul 21 13:52:57.815318 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.815275 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjh2c\" (UniqueName: \"kubernetes.io/projected/e2c0c579-6654-42c3-8164-f9af35a49535-kube-api-access-fjh2c\") pod \"e2c0c579-6654-42c3-8164-f9af35a49535\" (UID: \"e2c0c579-6654-42c3-8164-f9af35a49535\") " Jul 21 13:52:57.817426 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.817401 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2c0c579-6654-42c3-8164-f9af35a49535-kube-api-access-fjh2c" (OuterVolumeSpecName: "kube-api-access-fjh2c") pod "e2c0c579-6654-42c3-8164-f9af35a49535" (UID: "e2c0c579-6654-42c3-8164-f9af35a49535"). InnerVolumeSpecName "kube-api-access-fjh2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:52:57.916514 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:57.916419 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjh2c\" (UniqueName: \"kubernetes.io/projected/e2c0c579-6654-42c3-8164-f9af35a49535-kube-api-access-fjh2c\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:52:58.339910 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:58.339819 2567 generic.go:356] "Generic (PLEG): container finished" podID="e2c0c579-6654-42c3-8164-f9af35a49535" containerID="8ac721da5035964aa1d8dfd68c1a4243c08359a214fdc59e0ce948e3f7da0e95" exitCode=0 Jul 21 13:52:58.339910 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:58.339862 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mmvnc" Jul 21 13:52:58.339910 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:58.339903 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mmvnc" event={"ID":"e2c0c579-6654-42c3-8164-f9af35a49535","Type":"ContainerDied","Data":"8ac721da5035964aa1d8dfd68c1a4243c08359a214fdc59e0ce948e3f7da0e95"} Jul 21 13:52:58.340163 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:58.339938 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mmvnc" event={"ID":"e2c0c579-6654-42c3-8164-f9af35a49535","Type":"ContainerDied","Data":"33eaac487b2a7255e3a04d7305b8abca95c6c8ff4b9ebed53feff20d1b5e22b1"} Jul 21 13:52:58.340163 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:58.339961 2567 scope.go:117] "RemoveContainer" containerID="8ac721da5035964aa1d8dfd68c1a4243c08359a214fdc59e0ce948e3f7da0e95" Jul 21 13:52:58.349614 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:58.349592 2567 scope.go:117] "RemoveContainer" containerID="8ac721da5035964aa1d8dfd68c1a4243c08359a214fdc59e0ce948e3f7da0e95" Jul 21 13:52:58.349897 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:52:58.349878 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ac721da5035964aa1d8dfd68c1a4243c08359a214fdc59e0ce948e3f7da0e95\": container with ID starting with 8ac721da5035964aa1d8dfd68c1a4243c08359a214fdc59e0ce948e3f7da0e95 not found: ID does not exist" containerID="8ac721da5035964aa1d8dfd68c1a4243c08359a214fdc59e0ce948e3f7da0e95" Jul 21 13:52:58.349981 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:58.349904 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ac721da5035964aa1d8dfd68c1a4243c08359a214fdc59e0ce948e3f7da0e95"} err="failed to get container status \"8ac721da5035964aa1d8dfd68c1a4243c08359a214fdc59e0ce948e3f7da0e95\": rpc error: code = NotFound desc = could not find container \"8ac721da5035964aa1d8dfd68c1a4243c08359a214fdc59e0ce948e3f7da0e95\": container with ID starting with 8ac721da5035964aa1d8dfd68c1a4243c08359a214fdc59e0ce948e3f7da0e95 not found: ID does not exist" Jul 21 13:52:58.363924 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:58.363895 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmvnc"] Jul 21 13:52:58.368663 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:58.368639 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmvnc"] Jul 21 13:52:59.532898 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:52:59.532866 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2c0c579-6654-42c3-8164-f9af35a49535" path="/var/lib/kubelet/pods/e2c0c579-6654-42c3-8164-f9af35a49535/volumes" Jul 21 13:53:02.176546 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:02.176501 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:02.337153 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:02.337117 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:02.542944 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:02.542848 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:06.139026 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:06.138985 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:06.304924 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:06.304882 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:22.899082 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:22.899043 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:22.994403 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:22.994364 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:30.769173 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:30.769132 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:30.840373 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:30.840328 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:39.260185 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:39.260137 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:39.443108 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:39.443074 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:39.997945 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:39.997865 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:40.188357 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:40.188316 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:40.770871 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:40.770836 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:40.957668 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:40.957621 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:40.987288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:40.987256 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-f45qg"] Jul 21 13:53:40.987630 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:40.987617 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2c0c579-6654-42c3-8164-f9af35a49535" containerName="authorino" Jul 21 13:53:40.987630 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:40.987631 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2c0c579-6654-42c3-8164-f9af35a49535" containerName="authorino" Jul 21 13:53:40.987750 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:40.987638 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d26f7052-d54f-4e0e-8063-4571da7db18f" containerName="authorino" Jul 21 13:53:40.987750 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:40.987643 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d26f7052-d54f-4e0e-8063-4571da7db18f" containerName="authorino" Jul 21 13:53:40.987750 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:40.987722 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d26f7052-d54f-4e0e-8063-4571da7db18f" containerName="authorino" Jul 21 13:53:40.987750 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:40.987732 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2c0c579-6654-42c3-8164-f9af35a49535" containerName="authorino" Jul 21 13:53:40.990982 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:40.990963 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-f45qg" Jul 21 13:53:40.994677 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:40.994655 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 21 13:53:40.995985 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:40.995969 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jdr9h\"" Jul 21 13:53:41.007732 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:41.007684 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-f45qg"] Jul 21 13:53:41.105511 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:41.105429 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scrkb\" (UniqueName: \"kubernetes.io/projected/f668a1ed-b892-4b42-9151-66990c6511e4-kube-api-access-scrkb\") pod \"payload-processing-5bf47cd988-f45qg\" (UID: \"f668a1ed-b892-4b42-9151-66990c6511e4\") " pod="openshift-ingress/payload-processing-5bf47cd988-f45qg" Jul 21 13:53:41.105511 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:41.105486 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f668a1ed-b892-4b42-9151-66990c6511e4-plugins-config-volume\") pod \"payload-processing-5bf47cd988-f45qg\" (UID: \"f668a1ed-b892-4b42-9151-66990c6511e4\") " pod="openshift-ingress/payload-processing-5bf47cd988-f45qg" Jul 21 13:53:41.206051 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:41.206009 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scrkb\" (UniqueName: \"kubernetes.io/projected/f668a1ed-b892-4b42-9151-66990c6511e4-kube-api-access-scrkb\") pod \"payload-processing-5bf47cd988-f45qg\" (UID: \"f668a1ed-b892-4b42-9151-66990c6511e4\") " pod="openshift-ingress/payload-processing-5bf47cd988-f45qg" Jul 21 13:53:41.206231 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:41.206065 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f668a1ed-b892-4b42-9151-66990c6511e4-plugins-config-volume\") pod \"payload-processing-5bf47cd988-f45qg\" (UID: \"f668a1ed-b892-4b42-9151-66990c6511e4\") " pod="openshift-ingress/payload-processing-5bf47cd988-f45qg" Jul 21 13:53:41.206839 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:41.206818 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f668a1ed-b892-4b42-9151-66990c6511e4-plugins-config-volume\") pod \"payload-processing-5bf47cd988-f45qg\" (UID: \"f668a1ed-b892-4b42-9151-66990c6511e4\") " pod="openshift-ingress/payload-processing-5bf47cd988-f45qg" Jul 21 13:53:41.215602 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:41.215570 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scrkb\" (UniqueName: \"kubernetes.io/projected/f668a1ed-b892-4b42-9151-66990c6511e4-kube-api-access-scrkb\") pod \"payload-processing-5bf47cd988-f45qg\" (UID: \"f668a1ed-b892-4b42-9151-66990c6511e4\") " pod="openshift-ingress/payload-processing-5bf47cd988-f45qg" Jul 21 13:53:41.301560 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:41.301523 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-f45qg" Jul 21 13:53:41.448046 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:41.448014 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-f45qg"] Jul 21 13:53:41.455816 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:53:41.455776 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf668a1ed_b892_4b42_9151_66990c6511e4.slice/crio-189ebc438122e322939f188deb6b30e25f3372f218389dc800cd202039c6018a WatchSource:0}: Error finding container 189ebc438122e322939f188deb6b30e25f3372f218389dc800cd202039c6018a: Status 404 returned error can't find the container with id 189ebc438122e322939f188deb6b30e25f3372f218389dc800cd202039c6018a Jul 21 13:53:41.581995 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:41.581954 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-f45qg" event={"ID":"f668a1ed-b892-4b42-9151-66990c6511e4","Type":"ContainerStarted","Data":"189ebc438122e322939f188deb6b30e25f3372f218389dc800cd202039c6018a"} Jul 21 13:53:43.597644 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:43.597599 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-f45qg" event={"ID":"f668a1ed-b892-4b42-9151-66990c6511e4","Type":"ContainerStarted","Data":"896cea7a03006d8992a003822896ca2916e0a1b820529b9e7bfc5b993f075c5d"} Jul 21 13:53:43.598033 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:43.597722 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-f45qg" Jul 21 13:53:48.628758 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:48.628665 2567 generic.go:356] "Generic (PLEG): container finished" podID="5a886230-716b-4ef6-97ae-0847508d4ab7" containerID="f66f86bb71548d9c7ccb18401186c4c9da786ed67e939a22c3b3a35b5280c530" exitCode=0 Jul 21 13:53:48.628758 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:48.628745 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" event={"ID":"5a886230-716b-4ef6-97ae-0847508d4ab7","Type":"ContainerDied","Data":"f66f86bb71548d9c7ccb18401186c4c9da786ed67e939a22c3b3a35b5280c530"} Jul 21 13:53:48.629361 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:48.629152 2567 scope.go:117] "RemoveContainer" containerID="f66f86bb71548d9c7ccb18401186c4c9da786ed67e939a22c3b3a35b5280c530" Jul 21 13:53:48.630225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:48.630199 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 13:53:48.630324 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:48.630283 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 13:53:48.630375 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:48.630338 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 13:53:48.678620 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:48.678573 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-f45qg" podStartSLOduration=6.7014967500000004 podStartE2EDuration="8.678556691s" totalImagesPullingTime="1.977059941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:53:43.627886031 +0000 UTC m=+584.779727141" watchObservedRunningTime="2026-07-21 13:53:48.678556691 +0000 UTC m=+589.830397850" Jul 21 13:53:49.632556 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:49.632489 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:49.642363 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:49.642332 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" event={"ID":"5a886230-716b-4ef6-97ae-0847508d4ab7","Type":"ContainerStarted","Data":"68f58f999fabdfcf4c7009b121fb68d7eadc608a9eaf008432d80d15d2977f24"} Jul 21 13:53:49.798953 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:49.798916 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:53:49.867253 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:49.867208 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:53:49.868441 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:49.868418 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-74949dcd89-dlm5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 21 13:53:49.868495 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:49.868468 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" podUID="5a886230-716b-4ef6-97ae-0847508d4ab7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 21 13:53:50.867392 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:50.867342 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-74949dcd89-dlm5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 21 13:53:50.867810 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:50.867416 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" podUID="5a886230-716b-4ef6-97ae-0847508d4ab7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 21 13:53:51.867843 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:51.867810 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-74949dcd89-dlm5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 21 13:53:51.868215 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:51.867864 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" podUID="5a886230-716b-4ef6-97ae-0847508d4ab7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 21 13:53:52.124911 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:52.124800 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:53:52.867739 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:52.867678 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-74949dcd89-dlm5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 21 13:53:52.867932 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:52.867768 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" podUID="5a886230-716b-4ef6-97ae-0847508d4ab7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 21 13:53:53.867198 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:53.867160 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-74949dcd89-dlm5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 21 13:53:53.867447 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:53.867218 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" podUID="5a886230-716b-4ef6-97ae-0847508d4ab7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 21 13:53:54.605805 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:54.605766 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-f45qg" Jul 21 13:53:54.871449 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:54.871349 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:53:54.872485 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:54.872456 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74949dcd89-dlm5x" Jul 21 13:53:59.336318 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:59.336283 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 13:53:59.339371 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:53:59.339342 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 13:54:02.995063 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:02.995021 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:54:15.650039 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:15.649998 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:54:20.038035 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.037998 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr"] Jul 21 13:54:20.041646 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.041629 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.046210 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.046190 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 21 13:54:20.047403 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.047381 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 13:54:20.047539 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.047450 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 13:54:20.047794 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.047770 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gr2dk\"" Jul 21 13:54:20.063908 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.063886 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f0685b3-e171-4e33-ab1e-bf64414d5e68-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.064011 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.063922 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f0685b3-e171-4e33-ab1e-bf64414d5e68-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.064011 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.063945 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df4xl\" (UniqueName: \"kubernetes.io/projected/0f0685b3-e171-4e33-ab1e-bf64414d5e68-kube-api-access-df4xl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.064122 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.064054 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f0685b3-e171-4e33-ab1e-bf64414d5e68-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.064182 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.064119 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f0685b3-e171-4e33-ab1e-bf64414d5e68-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.064235 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.064179 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f0685b3-e171-4e33-ab1e-bf64414d5e68-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.071336 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.071314 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr"] Jul 21 13:54:20.083395 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.083374 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:54:20.165589 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.165564 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f0685b3-e171-4e33-ab1e-bf64414d5e68-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.165762 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.165600 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df4xl\" (UniqueName: \"kubernetes.io/projected/0f0685b3-e171-4e33-ab1e-bf64414d5e68-kube-api-access-df4xl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.165762 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.165628 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f0685b3-e171-4e33-ab1e-bf64414d5e68-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.165762 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.165655 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f0685b3-e171-4e33-ab1e-bf64414d5e68-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.165944 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.165766 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f0685b3-e171-4e33-ab1e-bf64414d5e68-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.165944 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.165827 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f0685b3-e171-4e33-ab1e-bf64414d5e68-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.166060 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.166038 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f0685b3-e171-4e33-ab1e-bf64414d5e68-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.166099 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.166053 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f0685b3-e171-4e33-ab1e-bf64414d5e68-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.166157 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.166134 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f0685b3-e171-4e33-ab1e-bf64414d5e68-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.167918 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.167897 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f0685b3-e171-4e33-ab1e-bf64414d5e68-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.168191 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.168172 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f0685b3-e171-4e33-ab1e-bf64414d5e68-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.174157 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.174139 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df4xl\" (UniqueName: \"kubernetes.io/projected/0f0685b3-e171-4e33-ab1e-bf64414d5e68-kube-api-access-df4xl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-f2bnr\" (UID: \"0f0685b3-e171-4e33-ab1e-bf64414d5e68\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.352503 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.352420 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:20.495263 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.495237 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr"] Jul 21 13:54:20.497269 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:54:20.497244 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0685b3_e171_4e33_ab1e_bf64414d5e68.slice/crio-0bed935c6d79ac04d1e9673e375668fdc18b2a8fa190400b705cfd31647c7245 WatchSource:0}: Error finding container 0bed935c6d79ac04d1e9673e375668fdc18b2a8fa190400b705cfd31647c7245: Status 404 returned error can't find the container with id 0bed935c6d79ac04d1e9673e375668fdc18b2a8fa190400b705cfd31647c7245 Jul 21 13:54:20.498500 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.498477 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 13:54:20.821581 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:20.821543 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" event={"ID":"0f0685b3-e171-4e33-ab1e-bf64414d5e68","Type":"ContainerStarted","Data":"0bed935c6d79ac04d1e9673e375668fdc18b2a8fa190400b705cfd31647c7245"} Jul 21 13:54:21.154358 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:21.154321 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:54:21.250954 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:21.250866 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:54:22.834751 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:22.834684 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" event={"ID":"0f0685b3-e171-4e33-ab1e-bf64414d5e68","Type":"ContainerStarted","Data":"60fbd06ee5714c5ac921df9392b70fe912a5914c8cb0a22cf2b7247000bbe20b"} Jul 21 13:54:22.856336 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:22.856275 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" podStartSLOduration=1.402855228 podStartE2EDuration="2.856257723s" totalImagesPullingTime="1.453402495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:54:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:54:22.855334258 +0000 UTC m=+624.007175366" watchObservedRunningTime="2026-07-21 13:54:22.856257723 +0000 UTC m=+624.008098832" Jul 21 13:54:26.774278 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:26.774243 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:54:26.998001 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:26.997969 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:54:30.353233 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:30.353196 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:30.353599 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:30.353266 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:30.365678 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:30.365649 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:30.887778 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:30.887749 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-f2bnr" Jul 21 13:54:31.032099 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:31.032069 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:54:46.066317 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:46.066273 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:54:49.590360 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.590301 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp"] Jul 21 13:54:49.597180 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.597148 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.600084 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.600058 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 21 13:54:49.604738 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.604690 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp"] Jul 21 13:54:49.616359 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.616314 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:54:49.748733 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.748663 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.748949 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.748777 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.748949 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.748837 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.748949 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.748877 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g275h\" (UniqueName: \"kubernetes.io/projected/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-kube-api-access-g275h\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.749099 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.748963 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.749099 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.749023 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.850647 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.850550 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.850647 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.850603 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g275h\" (UniqueName: \"kubernetes.io/projected/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-kube-api-access-g275h\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.850647 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.850632 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.851012 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.850667 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.851012 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.850749 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.851012 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.850801 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.851145 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.851013 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.851145 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.851133 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.851250 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.851214 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.853293 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.853266 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.853546 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.853528 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.861912 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.861878 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g275h\" (UniqueName: \"kubernetes.io/projected/c40bffa9-57b8-4427-8aa7-96ac6a4e8557-kube-api-access-g275h\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp\" (UID: \"c40bffa9-57b8-4427-8aa7-96ac6a4e8557\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:49.910939 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:49.910886 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:50.130288 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:50.130247 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp"] Jul 21 13:54:50.132298 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:54:50.132261 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc40bffa9_57b8_4427_8aa7_96ac6a4e8557.slice/crio-40872275dff807e4bb89a158b960bb64588164c9a440e1ae67de1515ad558fe7 WatchSource:0}: Error finding container 40872275dff807e4bb89a158b960bb64588164c9a440e1ae67de1515ad558fe7: Status 404 returned error can't find the container with id 40872275dff807e4bb89a158b960bb64588164c9a440e1ae67de1515ad558fe7 Jul 21 13:54:50.995189 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:50.995138 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" event={"ID":"c40bffa9-57b8-4427-8aa7-96ac6a4e8557","Type":"ContainerStarted","Data":"f67abfb99197398751d46a28a7c853825c0e8d2e7f72dd8edb870640bdb92227"} Jul 21 13:54:50.995657 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:50.995201 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" event={"ID":"c40bffa9-57b8-4427-8aa7-96ac6a4e8557","Type":"ContainerStarted","Data":"40872275dff807e4bb89a158b960bb64588164c9a440e1ae67de1515ad558fe7"} Jul 21 13:54:51.029573 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:51.029509 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" podStartSLOduration=1.777653205 podStartE2EDuration="2.029485182s" totalImagesPullingTime="251.831977ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:54:51.028659006 +0000 UTC m=+652.180500116" watchObservedRunningTime="2026-07-21 13:54:51.029485182 +0000 UTC m=+652.181326288" Jul 21 13:54:55.211338 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:55.211297 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:54:59.911061 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:59.911013 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:59.911061 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:59.911070 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:54:59.924264 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:54:59.924226 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:55:00.056976 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:00.056944 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp" Jul 21 13:55:02.349280 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:02.349239 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:55:31.368384 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:31.368344 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78bf958d9-8ndr2"] Jul 21 13:55:31.372021 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:31.371998 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78bf958d9-8ndr2" Jul 21 13:55:31.381885 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:31.381861 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78bf958d9-8ndr2"] Jul 21 13:55:31.427752 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:31.427719 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dppt\" (UniqueName: \"kubernetes.io/projected/cf8adfce-4ec2-4552-8c9d-16bdeb50dfab-kube-api-access-5dppt\") pod \"authorino-78bf958d9-8ndr2\" (UID: \"cf8adfce-4ec2-4552-8c9d-16bdeb50dfab\") " pod="kuadrant-system/authorino-78bf958d9-8ndr2" Jul 21 13:55:31.427909 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:31.427835 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cf8adfce-4ec2-4552-8c9d-16bdeb50dfab-tls-cert\") pod \"authorino-78bf958d9-8ndr2\" (UID: \"cf8adfce-4ec2-4552-8c9d-16bdeb50dfab\") " pod="kuadrant-system/authorino-78bf958d9-8ndr2" Jul 21 13:55:31.529092 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:31.529051 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cf8adfce-4ec2-4552-8c9d-16bdeb50dfab-tls-cert\") pod \"authorino-78bf958d9-8ndr2\" (UID: \"cf8adfce-4ec2-4552-8c9d-16bdeb50dfab\") " pod="kuadrant-system/authorino-78bf958d9-8ndr2" Jul 21 13:55:31.529282 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:31.529101 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dppt\" (UniqueName: \"kubernetes.io/projected/cf8adfce-4ec2-4552-8c9d-16bdeb50dfab-kube-api-access-5dppt\") pod \"authorino-78bf958d9-8ndr2\" (UID: \"cf8adfce-4ec2-4552-8c9d-16bdeb50dfab\") " pod="kuadrant-system/authorino-78bf958d9-8ndr2" Jul 21 13:55:31.531598 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:31.531574 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cf8adfce-4ec2-4552-8c9d-16bdeb50dfab-tls-cert\") pod \"authorino-78bf958d9-8ndr2\" (UID: \"cf8adfce-4ec2-4552-8c9d-16bdeb50dfab\") " pod="kuadrant-system/authorino-78bf958d9-8ndr2" Jul 21 13:55:31.539987 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:31.539964 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dppt\" (UniqueName: \"kubernetes.io/projected/cf8adfce-4ec2-4552-8c9d-16bdeb50dfab-kube-api-access-5dppt\") pod \"authorino-78bf958d9-8ndr2\" (UID: \"cf8adfce-4ec2-4552-8c9d-16bdeb50dfab\") " pod="kuadrant-system/authorino-78bf958d9-8ndr2" Jul 21 13:55:31.681815 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:31.681785 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78bf958d9-8ndr2" Jul 21 13:55:31.823675 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:31.823636 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78bf958d9-8ndr2"] Jul 21 13:55:31.826432 ip-10-0-129-212 kubenswrapper[2567]: W0721 13:55:31.826409 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf8adfce_4ec2_4552_8c9d_16bdeb50dfab.slice/crio-99fb67aeb202f13f313305e23b9d8645011474bd0284045c4a43b0b2ee37e6a1 WatchSource:0}: Error finding container 99fb67aeb202f13f313305e23b9d8645011474bd0284045c4a43b0b2ee37e6a1: Status 404 returned error can't find the container with id 99fb67aeb202f13f313305e23b9d8645011474bd0284045c4a43b0b2ee37e6a1 Jul 21 13:55:31.842365 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:31.842340 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:55:31.943934 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:31.943843 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:55:32.227255 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:32.227172 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78bf958d9-8ndr2" event={"ID":"cf8adfce-4ec2-4552-8c9d-16bdeb50dfab","Type":"ContainerStarted","Data":"99fb67aeb202f13f313305e23b9d8645011474bd0284045c4a43b0b2ee37e6a1"} Jul 21 13:55:33.235885 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:33.235837 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78bf958d9-8ndr2" event={"ID":"cf8adfce-4ec2-4552-8c9d-16bdeb50dfab","Type":"ContainerStarted","Data":"3fe8374ea4b5b11aaf745a31803c131d2db606106bbc2db0b0ebaaebd693ce0a"} Jul 21 13:55:33.285276 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:33.285208 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-78bf958d9-8ndr2" podStartSLOduration=1.464498288 podStartE2EDuration="2.285187711s" totalImagesPullingTime="820.689423ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:55:33.278542509 +0000 UTC m=+694.430383619" watchObservedRunningTime="2026-07-21 13:55:33.285187711 +0000 UTC m=+694.437028821" Jul 21 13:55:33.304225 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:33.304180 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:55:33.401546 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:33.401513 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9978846dd-qc989"] Jul 21 13:55:33.401728 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:33.401682 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-9978846dd-qc989" podUID="ba660e25-8f20-49cb-9675-26a628a75067" containerName="authorino" containerID="cri-o://61187c881fca7afaa62ba739fec664f00bef34a77335c273d736fc0a1ff2a083" gracePeriod=30 Jul 21 13:55:33.464770 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:33.464738 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:55:33.685313 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:33.685283 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9978846dd-qc989" Jul 21 13:55:33.748189 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:33.748160 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba660e25-8f20-49cb-9675-26a628a75067-tls-cert\") pod \"ba660e25-8f20-49cb-9675-26a628a75067\" (UID: \"ba660e25-8f20-49cb-9675-26a628a75067\") " Jul 21 13:55:33.748381 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:33.748247 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w97sg\" (UniqueName: \"kubernetes.io/projected/ba660e25-8f20-49cb-9675-26a628a75067-kube-api-access-w97sg\") pod \"ba660e25-8f20-49cb-9675-26a628a75067\" (UID: \"ba660e25-8f20-49cb-9675-26a628a75067\") " Jul 21 13:55:33.750402 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:33.750366 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba660e25-8f20-49cb-9675-26a628a75067-kube-api-access-w97sg" (OuterVolumeSpecName: "kube-api-access-w97sg") pod "ba660e25-8f20-49cb-9675-26a628a75067" (UID: "ba660e25-8f20-49cb-9675-26a628a75067"). InnerVolumeSpecName "kube-api-access-w97sg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:55:33.761218 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:33.761190 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba660e25-8f20-49cb-9675-26a628a75067-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ba660e25-8f20-49cb-9675-26a628a75067" (UID: "ba660e25-8f20-49cb-9675-26a628a75067"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:55:33.849180 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:33.849095 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w97sg\" (UniqueName: \"kubernetes.io/projected/ba660e25-8f20-49cb-9675-26a628a75067-kube-api-access-w97sg\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:55:33.849180 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:33.849129 2567 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba660e25-8f20-49cb-9675-26a628a75067-tls-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 13:55:34.244556 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:34.244521 2567 generic.go:356] "Generic (PLEG): container finished" podID="ba660e25-8f20-49cb-9675-26a628a75067" containerID="61187c881fca7afaa62ba739fec664f00bef34a77335c273d736fc0a1ff2a083" exitCode=0 Jul 21 13:55:34.245045 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:34.244561 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9978846dd-qc989" Jul 21 13:55:34.245045 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:34.244611 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9978846dd-qc989" event={"ID":"ba660e25-8f20-49cb-9675-26a628a75067","Type":"ContainerDied","Data":"61187c881fca7afaa62ba739fec664f00bef34a77335c273d736fc0a1ff2a083"} Jul 21 13:55:34.245045 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:34.244660 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9978846dd-qc989" event={"ID":"ba660e25-8f20-49cb-9675-26a628a75067","Type":"ContainerDied","Data":"87ac0a0759c67f7222e0817418ea31ee0221d6bdc28719da4cee9921edc26c47"} Jul 21 13:55:34.245045 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:34.244685 2567 scope.go:117] "RemoveContainer" containerID="61187c881fca7afaa62ba739fec664f00bef34a77335c273d736fc0a1ff2a083" Jul 21 13:55:34.255533 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:34.255502 2567 scope.go:117] "RemoveContainer" containerID="61187c881fca7afaa62ba739fec664f00bef34a77335c273d736fc0a1ff2a083" Jul 21 13:55:34.255826 ip-10-0-129-212 kubenswrapper[2567]: E0721 13:55:34.255807 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61187c881fca7afaa62ba739fec664f00bef34a77335c273d736fc0a1ff2a083\": container with ID starting with 61187c881fca7afaa62ba739fec664f00bef34a77335c273d736fc0a1ff2a083 not found: ID does not exist" containerID="61187c881fca7afaa62ba739fec664f00bef34a77335c273d736fc0a1ff2a083" Jul 21 13:55:34.255895 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:34.255834 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61187c881fca7afaa62ba739fec664f00bef34a77335c273d736fc0a1ff2a083"} err="failed to get container status \"61187c881fca7afaa62ba739fec664f00bef34a77335c273d736fc0a1ff2a083\": rpc error: code = NotFound desc = could not find container \"61187c881fca7afaa62ba739fec664f00bef34a77335c273d736fc0a1ff2a083\": container with ID starting with 61187c881fca7afaa62ba739fec664f00bef34a77335c273d736fc0a1ff2a083 not found: ID does not exist" Jul 21 13:55:34.275142 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:34.275105 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-9978846dd-qc989"] Jul 21 13:55:34.284140 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:34.284115 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-9978846dd-qc989"] Jul 21 13:55:35.532157 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:55:35.532107 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba660e25-8f20-49cb-9675-26a628a75067" path="/var/lib/kubelet/pods/ba660e25-8f20-49cb-9675-26a628a75067/volumes" Jul 21 13:58:03.469273 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:58:03.469232 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:58:04.625935 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:58:04.625892 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:58:45.345247 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:58:45.345209 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:58:45.450283 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:58:45.450231 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:58:45.543916 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:58:45.543883 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:58:53.575802 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:58:53.575680 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:58:53.677274 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:58:53.677234 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:58:58.462864 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:58:58.462796 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:58:58.561375 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:58:58.561295 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:58:59.246920 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:58:59.246862 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 13:58:59.379843 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:58:59.379812 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 13:58:59.379843 ip-10-0-129-212 kubenswrapper[2567]: I0721 13:58:59.379833 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:00:16.145849 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:00:16.145806 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:01:03.018865 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:01:03.018757 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:02:57.749430 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:57.749386 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6"] Jul 21 14:02:57.750064 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:57.749635 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" podUID="1848a5d0-54ec-4255-a1ef-4a2f0a7bd059" containerName="manager" containerID="cri-o://437e92245fe2fcacc84ba700335464ce3138d65f6b53f345bfb703f807b11d91" gracePeriod=10 Jul 21 14:02:58.826943 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.826907 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" Jul 21 14:02:58.828982 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.828947 2567 generic.go:356] "Generic (PLEG): container finished" podID="1848a5d0-54ec-4255-a1ef-4a2f0a7bd059" containerID="437e92245fe2fcacc84ba700335464ce3138d65f6b53f345bfb703f807b11d91" exitCode=0 Jul 21 14:02:58.829185 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.828993 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" Jul 21 14:02:58.829185 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.829024 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" event={"ID":"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059","Type":"ContainerDied","Data":"437e92245fe2fcacc84ba700335464ce3138d65f6b53f345bfb703f807b11d91"} Jul 21 14:02:58.829185 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.829093 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6" event={"ID":"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059","Type":"ContainerDied","Data":"93f492028ab3e64782043ac2c1debddfffe6ee3ff12b0b3b30bac13792569504"} Jul 21 14:02:58.829185 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.829125 2567 scope.go:117] "RemoveContainer" containerID="437e92245fe2fcacc84ba700335464ce3138d65f6b53f345bfb703f807b11d91" Jul 21 14:02:58.841147 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.841117 2567 scope.go:117] "RemoveContainer" containerID="437e92245fe2fcacc84ba700335464ce3138d65f6b53f345bfb703f807b11d91" Jul 21 14:02:58.841522 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:02:58.841498 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"437e92245fe2fcacc84ba700335464ce3138d65f6b53f345bfb703f807b11d91\": container with ID starting with 437e92245fe2fcacc84ba700335464ce3138d65f6b53f345bfb703f807b11d91 not found: ID does not exist" containerID="437e92245fe2fcacc84ba700335464ce3138d65f6b53f345bfb703f807b11d91" Jul 21 14:02:58.841587 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.841536 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"437e92245fe2fcacc84ba700335464ce3138d65f6b53f345bfb703f807b11d91"} err="failed to get container status \"437e92245fe2fcacc84ba700335464ce3138d65f6b53f345bfb703f807b11d91\": rpc error: code = NotFound desc = could not find container \"437e92245fe2fcacc84ba700335464ce3138d65f6b53f345bfb703f807b11d91\": container with ID starting with 437e92245fe2fcacc84ba700335464ce3138d65f6b53f345bfb703f807b11d91 not found: ID does not exist" Jul 21 14:02:58.854152 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.853590 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lg8v\" (UniqueName: \"kubernetes.io/projected/1848a5d0-54ec-4255-a1ef-4a2f0a7bd059-kube-api-access-2lg8v\") pod \"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059\" (UID: \"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059\") " Jul 21 14:02:58.854152 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.853676 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1848a5d0-54ec-4255-a1ef-4a2f0a7bd059-extensions-socket-volume\") pod \"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059\" (UID: \"1848a5d0-54ec-4255-a1ef-4a2f0a7bd059\") " Jul 21 14:02:58.854455 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.854409 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1848a5d0-54ec-4255-a1ef-4a2f0a7bd059-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1848a5d0-54ec-4255-a1ef-4a2f0a7bd059" (UID: "1848a5d0-54ec-4255-a1ef-4a2f0a7bd059"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:02:58.856780 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.856737 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1848a5d0-54ec-4255-a1ef-4a2f0a7bd059-kube-api-access-2lg8v" (OuterVolumeSpecName: "kube-api-access-2lg8v") pod "1848a5d0-54ec-4255-a1ef-4a2f0a7bd059" (UID: "1848a5d0-54ec-4255-a1ef-4a2f0a7bd059"). InnerVolumeSpecName "kube-api-access-2lg8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:02:58.955330 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.955214 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lg8v\" (UniqueName: \"kubernetes.io/projected/1848a5d0-54ec-4255-a1ef-4a2f0a7bd059-kube-api-access-2lg8v\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:02:58.955330 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:58.955256 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1848a5d0-54ec-4255-a1ef-4a2f0a7bd059-extensions-socket-volume\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:02:59.168593 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:59.168539 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6"] Jul 21 14:02:59.185403 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:59.185347 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kb2q6"] Jul 21 14:02:59.535112 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:02:59.535064 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1848a5d0-54ec-4255-a1ef-4a2f0a7bd059" path="/var/lib/kubelet/pods/1848a5d0-54ec-4255-a1ef-4a2f0a7bd059/volumes" Jul 21 14:03:58.916573 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:58.916530 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8"] Jul 21 14:03:58.919080 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:58.916976 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba660e25-8f20-49cb-9675-26a628a75067" containerName="authorino" Jul 21 14:03:58.919080 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:58.916992 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba660e25-8f20-49cb-9675-26a628a75067" containerName="authorino" Jul 21 14:03:58.919080 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:58.917021 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1848a5d0-54ec-4255-a1ef-4a2f0a7bd059" containerName="manager" Jul 21 14:03:58.919080 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:58.917027 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1848a5d0-54ec-4255-a1ef-4a2f0a7bd059" containerName="manager" Jul 21 14:03:58.919080 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:58.917095 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1848a5d0-54ec-4255-a1ef-4a2f0a7bd059" containerName="manager" Jul 21 14:03:58.919080 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:58.917104 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba660e25-8f20-49cb-9675-26a628a75067" containerName="authorino" Jul 21 14:03:58.920234 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:58.920211 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8" Jul 21 14:03:58.923439 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:58.923408 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-44l6z\"" Jul 21 14:03:58.934397 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:58.934356 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8"] Jul 21 14:03:59.033034 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:59.032986 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb1c7b36-935b-4a47-9e77-2b394a660b05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dntz8\" (UID: \"cb1c7b36-935b-4a47-9e77-2b394a660b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8" Jul 21 14:03:59.033230 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:59.033065 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29g5p\" (UniqueName: \"kubernetes.io/projected/cb1c7b36-935b-4a47-9e77-2b394a660b05-kube-api-access-29g5p\") pod \"kuadrant-operator-controller-manager-86878c769f-dntz8\" (UID: \"cb1c7b36-935b-4a47-9e77-2b394a660b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8" Jul 21 14:03:59.134118 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:59.134064 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb1c7b36-935b-4a47-9e77-2b394a660b05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dntz8\" (UID: \"cb1c7b36-935b-4a47-9e77-2b394a660b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8" Jul 21 14:03:59.134327 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:59.134153 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29g5p\" (UniqueName: \"kubernetes.io/projected/cb1c7b36-935b-4a47-9e77-2b394a660b05-kube-api-access-29g5p\") pod \"kuadrant-operator-controller-manager-86878c769f-dntz8\" (UID: \"cb1c7b36-935b-4a47-9e77-2b394a660b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8" Jul 21 14:03:59.134535 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:59.134509 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb1c7b36-935b-4a47-9e77-2b394a660b05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dntz8\" (UID: \"cb1c7b36-935b-4a47-9e77-2b394a660b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8" Jul 21 14:03:59.144126 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:59.144087 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29g5p\" (UniqueName: \"kubernetes.io/projected/cb1c7b36-935b-4a47-9e77-2b394a660b05-kube-api-access-29g5p\") pod \"kuadrant-operator-controller-manager-86878c769f-dntz8\" (UID: \"cb1c7b36-935b-4a47-9e77-2b394a660b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8" Jul 21 14:03:59.233046 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:59.232927 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8" Jul 21 14:03:59.402123 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:59.402086 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8"] Jul 21 14:03:59.404777 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:03:59.404737 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb1c7b36_935b_4a47_9e77_2b394a660b05.slice/crio-ea72b63a0aa5c764cf7e3c7ebb898faf57c544c56e6cde83ee9bef81da2095c7 WatchSource:0}: Error finding container ea72b63a0aa5c764cf7e3c7ebb898faf57c544c56e6cde83ee9bef81da2095c7: Status 404 returned error can't find the container with id ea72b63a0aa5c764cf7e3c7ebb898faf57c544c56e6cde83ee9bef81da2095c7 Jul 21 14:03:59.421300 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:59.421270 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:03:59.421606 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:03:59.421586 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:04:00.178775 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:04:00.178726 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8" event={"ID":"cb1c7b36-935b-4a47-9e77-2b394a660b05","Type":"ContainerStarted","Data":"f4c781a3e23c0b92de398cbd9d10cc4558239452c546fb69ef870dee96129c69"} Jul 21 14:04:00.179234 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:04:00.178800 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8" event={"ID":"cb1c7b36-935b-4a47-9e77-2b394a660b05","Type":"ContainerStarted","Data":"ea72b63a0aa5c764cf7e3c7ebb898faf57c544c56e6cde83ee9bef81da2095c7"} Jul 21 14:04:00.179234 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:04:00.178831 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8" Jul 21 14:04:00.210387 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:04:00.210326 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8" podStartSLOduration=2.210302174 podStartE2EDuration="2.210302174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:03:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:04:00.20795421 +0000 UTC m=+1201.359795318" watchObservedRunningTime="2026-07-21 14:04:00.210302174 +0000 UTC m=+1201.362143286" Jul 21 14:04:11.187663 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:04:11.187625 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dntz8" Jul 21 14:06:13.208207 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:13.208154 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:06:42.380325 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.380241 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv"] Jul 21 14:06:42.384400 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.384379 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.387211 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.387187 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-bb143a4b-kserve-self-signed-certs\"" Jul 21 14:06:42.398108 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.398081 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv"] Jul 21 14:06:42.466934 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.466872 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:06:42.541913 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.541872 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-home\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.541913 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.541915 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-tmp-dir\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.542161 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.541951 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-dshm\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.542161 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.542011 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/861275a7-188c-40eb-8d3c-ffe3c351b394-tls-certs\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.542161 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.542044 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-model-cache\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.542161 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.542105 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcjsz\" (UniqueName: \"kubernetes.io/projected/861275a7-188c-40eb-8d3c-ffe3c351b394-kube-api-access-jcjsz\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.642991 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.642891 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcjsz\" (UniqueName: \"kubernetes.io/projected/861275a7-188c-40eb-8d3c-ffe3c351b394-kube-api-access-jcjsz\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.642991 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.642966 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-home\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.642991 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.642982 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-tmp-dir\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.643301 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.643006 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-dshm\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.643301 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.643133 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/861275a7-188c-40eb-8d3c-ffe3c351b394-tls-certs\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.643301 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.643188 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-model-cache\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.643495 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.643460 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-home\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.643564 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.643526 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-tmp-dir\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.643624 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.643576 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-model-cache\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.645513 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.645493 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-dshm\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.645839 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.645812 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/861275a7-188c-40eb-8d3c-ffe3c351b394-tls-certs\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.650853 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.650832 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcjsz\" (UniqueName: \"kubernetes.io/projected/861275a7-188c-40eb-8d3c-ffe3c351b394-kube-api-access-jcjsz\") pod \"e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.694038 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.693997 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:42.840330 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:42.840301 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv"] Jul 21 14:06:42.842189 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:06:42.842162 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod861275a7_188c_40eb_8d3c_ffe3c351b394.slice/crio-f10286490c1b2161798e9c8e37d6a412fa383837656a4f3d67758be04d0139ce WatchSource:0}: Error finding container f10286490c1b2161798e9c8e37d6a412fa383837656a4f3d67758be04d0139ce: Status 404 returned error can't find the container with id f10286490c1b2161798e9c8e37d6a412fa383837656a4f3d67758be04d0139ce Jul 21 14:06:43.150891 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:43.150853 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" event={"ID":"861275a7-188c-40eb-8d3c-ffe3c351b394","Type":"ContainerStarted","Data":"310737cb32bffd4956f5a97809badf595ed63b4049da08eb7d4d256e4df13819"} Jul 21 14:06:43.150891 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:43.150899 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" event={"ID":"861275a7-188c-40eb-8d3c-ffe3c351b394","Type":"ContainerStarted","Data":"f10286490c1b2161798e9c8e37d6a412fa383837656a4f3d67758be04d0139ce"} Jul 21 14:06:43.168353 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:43.168297 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" podStartSLOduration=1.168278249 podStartE2EDuration="1.168278249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:06:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:06:43.167932678 +0000 UTC m=+1364.319773788" watchObservedRunningTime="2026-07-21 14:06:43.168278249 +0000 UTC m=+1364.320119357" Jul 21 14:06:52.695007 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:52.694958 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:52.695530 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:52.695087 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:52.708028 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:52.707990 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:53.225311 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:53.225280 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:06:58.170566 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.170488 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt"] Jul 21 14:06:58.174417 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.174389 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.177121 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.177097 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-bb143a4b-kserve-self-signed-certs\"" Jul 21 14:06:58.186002 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.185973 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt"] Jul 21 14:06:58.194056 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.194033 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-dshm\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.194163 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.194062 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-model-cache\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.194163 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.194155 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-home\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.194250 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.194184 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-tmp-dir\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.194250 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.194201 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrfs9\" (UniqueName: \"kubernetes.io/projected/965e6fb7-39e7-4a96-b887-df812112adbb-kube-api-access-rrfs9\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.194250 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.194218 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965e6fb7-39e7-4a96-b887-df812112adbb-tls-certs\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.213182 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.213151 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:06:58.294664 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.294627 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-home\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.294664 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.294664 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-tmp-dir\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.294950 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.294683 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrfs9\" (UniqueName: \"kubernetes.io/projected/965e6fb7-39e7-4a96-b887-df812112adbb-kube-api-access-rrfs9\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.294950 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.294734 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965e6fb7-39e7-4a96-b887-df812112adbb-tls-certs\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.294950 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.294794 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-dshm\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.294950 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.294816 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-model-cache\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.295172 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.295128 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-home\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.295172 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.295160 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-tmp-dir\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.295255 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.295240 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-model-cache\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.297131 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.297106 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-dshm\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.297378 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.297355 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965e6fb7-39e7-4a96-b887-df812112adbb-tls-certs\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.303508 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.303486 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrfs9\" (UniqueName: \"kubernetes.io/projected/965e6fb7-39e7-4a96-b887-df812112adbb-kube-api-access-rrfs9\") pod \"e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.486062 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.485953 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:06:58.638613 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:58.638580 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt"] Jul 21 14:06:58.639752 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:06:58.639722 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod965e6fb7_39e7_4a96_b887_df812112adbb.slice/crio-b244ffc6616f113745815900e8f81d68293b4ef042d72e9958b25c86381c448a WatchSource:0}: Error finding container b244ffc6616f113745815900e8f81d68293b4ef042d72e9958b25c86381c448a: Status 404 returned error can't find the container with id b244ffc6616f113745815900e8f81d68293b4ef042d72e9958b25c86381c448a Jul 21 14:06:59.250402 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:59.250363 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" event={"ID":"965e6fb7-39e7-4a96-b887-df812112adbb","Type":"ContainerStarted","Data":"88af9aa4cb3301191878116b0e5d24765bbb902d18e1c5f63afd8bd2065b1756"} Jul 21 14:06:59.250796 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:59.250412 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" event={"ID":"965e6fb7-39e7-4a96-b887-df812112adbb","Type":"ContainerStarted","Data":"b244ffc6616f113745815900e8f81d68293b4ef042d72e9958b25c86381c448a"} Jul 21 14:06:59.270614 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:06:59.270559 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" podStartSLOduration=1.270540728 podStartE2EDuration="1.270540728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:06:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:06:59.268172777 +0000 UTC m=+1380.420013889" watchObservedRunningTime="2026-07-21 14:06:59.270540728 +0000 UTC m=+1380.422381836" Jul 21 14:07:08.486360 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:08.486317 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:07:08.486804 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:08.486382 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:07:08.499075 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:08.499046 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:07:09.317377 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:09.317348 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:07:18.672720 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:18.672655 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:07:22.505484 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:22.505447 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv"] Jul 21 14:07:22.527235 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:22.527184 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:07:23.293658 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:23.293625 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt"] Jul 21 14:07:23.310057 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:23.310025 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:07:28.889099 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:28.889065 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:07:37.315284 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.315245 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:07:37.416019 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.415984 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:07:37.525865 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.525819 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" podUID="861275a7-188c-40eb-8d3c-ffe3c351b394" containerName="main" containerID="cri-o://310737cb32bffd4956f5a97809badf595ed63b4049da08eb7d4d256e4df13819" gracePeriod=45 Jul 21 14:07:37.803501 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.803466 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:07:37.870578 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.870401 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/861275a7-188c-40eb-8d3c-ffe3c351b394-tls-certs\") pod \"861275a7-188c-40eb-8d3c-ffe3c351b394\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " Jul 21 14:07:37.870578 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.870479 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-tmp-dir\") pod \"861275a7-188c-40eb-8d3c-ffe3c351b394\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " Jul 21 14:07:37.870578 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.870510 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-dshm\") pod \"861275a7-188c-40eb-8d3c-ffe3c351b394\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " Jul 21 14:07:37.870578 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.870559 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcjsz\" (UniqueName: \"kubernetes.io/projected/861275a7-188c-40eb-8d3c-ffe3c351b394-kube-api-access-jcjsz\") pod \"861275a7-188c-40eb-8d3c-ffe3c351b394\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " Jul 21 14:07:37.870981 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.870621 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-home\") pod \"861275a7-188c-40eb-8d3c-ffe3c351b394\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " Jul 21 14:07:37.870981 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.870674 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-model-cache\") pod \"861275a7-188c-40eb-8d3c-ffe3c351b394\" (UID: \"861275a7-188c-40eb-8d3c-ffe3c351b394\") " Jul 21 14:07:37.871206 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.871150 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "861275a7-188c-40eb-8d3c-ffe3c351b394" (UID: "861275a7-188c-40eb-8d3c-ffe3c351b394"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:07:37.871314 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.871250 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-model-cache" (OuterVolumeSpecName: "model-cache") pod "861275a7-188c-40eb-8d3c-ffe3c351b394" (UID: "861275a7-188c-40eb-8d3c-ffe3c351b394"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:07:37.871396 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.871330 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-home" (OuterVolumeSpecName: "home") pod "861275a7-188c-40eb-8d3c-ffe3c351b394" (UID: "861275a7-188c-40eb-8d3c-ffe3c351b394"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:07:37.873507 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.873471 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/861275a7-188c-40eb-8d3c-ffe3c351b394-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "861275a7-188c-40eb-8d3c-ffe3c351b394" (UID: "861275a7-188c-40eb-8d3c-ffe3c351b394"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 14:07:37.873637 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.873510 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/861275a7-188c-40eb-8d3c-ffe3c351b394-kube-api-access-jcjsz" (OuterVolumeSpecName: "kube-api-access-jcjsz") pod "861275a7-188c-40eb-8d3c-ffe3c351b394" (UID: "861275a7-188c-40eb-8d3c-ffe3c351b394"). InnerVolumeSpecName "kube-api-access-jcjsz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:07:37.873637 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.873559 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-dshm" (OuterVolumeSpecName: "dshm") pod "861275a7-188c-40eb-8d3c-ffe3c351b394" (UID: "861275a7-188c-40eb-8d3c-ffe3c351b394"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:07:37.972099 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.972058 2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-model-cache\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:07:37.972099 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.972098 2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/861275a7-188c-40eb-8d3c-ffe3c351b394-tls-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:07:37.972099 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.972110 2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-tmp-dir\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:07:37.972099 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.972123 2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-dshm\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:07:37.972424 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.972137 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcjsz\" (UniqueName: \"kubernetes.io/projected/861275a7-188c-40eb-8d3c-ffe3c351b394-kube-api-access-jcjsz\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:07:37.972424 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:37.972151 2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/861275a7-188c-40eb-8d3c-ffe3c351b394-home\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:07:38.314692 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.314552 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" podUID="965e6fb7-39e7-4a96-b887-df812112adbb" containerName="main" containerID="cri-o://88af9aa4cb3301191878116b0e5d24765bbb902d18e1c5f63afd8bd2065b1756" gracePeriod=45 Jul 21 14:07:38.483583 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.483547 2567 generic.go:356] "Generic (PLEG): container finished" podID="965e6fb7-39e7-4a96-b887-df812112adbb" containerID="88af9aa4cb3301191878116b0e5d24765bbb902d18e1c5f63afd8bd2065b1756" exitCode=0 Jul 21 14:07:38.484038 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.483638 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" event={"ID":"965e6fb7-39e7-4a96-b887-df812112adbb","Type":"ContainerDied","Data":"88af9aa4cb3301191878116b0e5d24765bbb902d18e1c5f63afd8bd2065b1756"} Jul 21 14:07:38.485196 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.485159 2567 generic.go:356] "Generic (PLEG): container finished" podID="861275a7-188c-40eb-8d3c-ffe3c351b394" containerID="310737cb32bffd4956f5a97809badf595ed63b4049da08eb7d4d256e4df13819" exitCode=0 Jul 21 14:07:38.485331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.485231 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" Jul 21 14:07:38.485331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.485255 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" event={"ID":"861275a7-188c-40eb-8d3c-ffe3c351b394","Type":"ContainerDied","Data":"310737cb32bffd4956f5a97809badf595ed63b4049da08eb7d4d256e4df13819"} Jul 21 14:07:38.485331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.485321 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv" event={"ID":"861275a7-188c-40eb-8d3c-ffe3c351b394","Type":"ContainerDied","Data":"f10286490c1b2161798e9c8e37d6a412fa383837656a4f3d67758be04d0139ce"} Jul 21 14:07:38.485432 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.485353 2567 scope.go:117] "RemoveContainer" containerID="310737cb32bffd4956f5a97809badf595ed63b4049da08eb7d4d256e4df13819" Jul 21 14:07:38.500603 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.500574 2567 scope.go:117] "RemoveContainer" containerID="310737cb32bffd4956f5a97809badf595ed63b4049da08eb7d4d256e4df13819" Jul 21 14:07:38.501587 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:07:38.501175 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"310737cb32bffd4956f5a97809badf595ed63b4049da08eb7d4d256e4df13819\": container with ID starting with 310737cb32bffd4956f5a97809badf595ed63b4049da08eb7d4d256e4df13819 not found: ID does not exist" containerID="310737cb32bffd4956f5a97809badf595ed63b4049da08eb7d4d256e4df13819" Jul 21 14:07:38.501587 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.501218 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"310737cb32bffd4956f5a97809badf595ed63b4049da08eb7d4d256e4df13819"} err="failed to get container status \"310737cb32bffd4956f5a97809badf595ed63b4049da08eb7d4d256e4df13819\": rpc error: code = NotFound desc = could not find container \"310737cb32bffd4956f5a97809badf595ed63b4049da08eb7d4d256e4df13819\": container with ID starting with 310737cb32bffd4956f5a97809badf595ed63b4049da08eb7d4d256e4df13819 not found: ID does not exist" Jul 21 14:07:38.520256 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.520214 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv"] Jul 21 14:07:38.524133 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.524098 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-bb143a4b-kserve-6848477899-g5fbv"] Jul 21 14:07:38.593998 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.593956 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:07:38.678373 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.678328 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-tmp-dir\") pod \"965e6fb7-39e7-4a96-b887-df812112adbb\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " Jul 21 14:07:38.678568 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.678438 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-model-cache\") pod \"965e6fb7-39e7-4a96-b887-df812112adbb\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " Jul 21 14:07:38.678568 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.678465 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrfs9\" (UniqueName: \"kubernetes.io/projected/965e6fb7-39e7-4a96-b887-df812112adbb-kube-api-access-rrfs9\") pod \"965e6fb7-39e7-4a96-b887-df812112adbb\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " Jul 21 14:07:38.678568 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.678487 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-dshm\") pod \"965e6fb7-39e7-4a96-b887-df812112adbb\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " Jul 21 14:07:38.678568 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.678532 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-home\") pod \"965e6fb7-39e7-4a96-b887-df812112adbb\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " Jul 21 14:07:38.678804 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.678572 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965e6fb7-39e7-4a96-b887-df812112adbb-tls-certs\") pod \"965e6fb7-39e7-4a96-b887-df812112adbb\" (UID: \"965e6fb7-39e7-4a96-b887-df812112adbb\") " Jul 21 14:07:38.678804 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.678665 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "965e6fb7-39e7-4a96-b887-df812112adbb" (UID: "965e6fb7-39e7-4a96-b887-df812112adbb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:07:38.678804 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.678782 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-model-cache" (OuterVolumeSpecName: "model-cache") pod "965e6fb7-39e7-4a96-b887-df812112adbb" (UID: "965e6fb7-39e7-4a96-b887-df812112adbb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:07:38.678972 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.678877 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-home" (OuterVolumeSpecName: "home") pod "965e6fb7-39e7-4a96-b887-df812112adbb" (UID: "965e6fb7-39e7-4a96-b887-df812112adbb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:07:38.678972 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.678900 2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-tmp-dir\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:07:38.678972 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.678919 2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-model-cache\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:07:38.680965 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.680927 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-dshm" (OuterVolumeSpecName: "dshm") pod "965e6fb7-39e7-4a96-b887-df812112adbb" (UID: "965e6fb7-39e7-4a96-b887-df812112adbb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:07:38.681112 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.680985 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/965e6fb7-39e7-4a96-b887-df812112adbb-kube-api-access-rrfs9" (OuterVolumeSpecName: "kube-api-access-rrfs9") pod "965e6fb7-39e7-4a96-b887-df812112adbb" (UID: "965e6fb7-39e7-4a96-b887-df812112adbb"). InnerVolumeSpecName "kube-api-access-rrfs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:07:38.681112 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.681020 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/965e6fb7-39e7-4a96-b887-df812112adbb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "965e6fb7-39e7-4a96-b887-df812112adbb" (UID: "965e6fb7-39e7-4a96-b887-df812112adbb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 14:07:38.779679 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.779634 2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/965e6fb7-39e7-4a96-b887-df812112adbb-tls-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:07:38.779679 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.779675 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrfs9\" (UniqueName: \"kubernetes.io/projected/965e6fb7-39e7-4a96-b887-df812112adbb-kube-api-access-rrfs9\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:07:38.779679 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.779687 2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-dshm\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:07:38.779961 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:38.779731 2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/965e6fb7-39e7-4a96-b887-df812112adbb-home\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:07:39.492571 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:39.492535 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" Jul 21 14:07:39.492571 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:39.492553 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt" event={"ID":"965e6fb7-39e7-4a96-b887-df812112adbb","Type":"ContainerDied","Data":"b244ffc6616f113745815900e8f81d68293b4ef042d72e9958b25c86381c448a"} Jul 21 14:07:39.493237 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:39.492614 2567 scope.go:117] "RemoveContainer" containerID="88af9aa4cb3301191878116b0e5d24765bbb902d18e1c5f63afd8bd2065b1756" Jul 21 14:07:39.517694 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:39.517641 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt"] Jul 21 14:07:39.520684 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:39.520657 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-bb143a4b-kserve-9c7fd79f5-ldtnt"] Jul 21 14:07:39.533642 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:39.533607 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="861275a7-188c-40eb-8d3c-ffe3c351b394" path="/var/lib/kubelet/pods/861275a7-188c-40eb-8d3c-ffe3c351b394/volumes" Jul 21 14:07:39.534124 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:07:39.534107 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="965e6fb7-39e7-4a96-b887-df812112adbb" path="/var/lib/kubelet/pods/965e6fb7-39e7-4a96-b887-df812112adbb/volumes" Jul 21 14:08:59.463841 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:08:59.463812 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:08:59.466330 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:08:59.465415 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:10:02.740130 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:02.740028 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:21.754778 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:21.754735 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:39.514109 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:39.514038 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:39.607679 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:39.607640 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:39.957658 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:39.957621 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:46.182441 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.182396 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:46.286038 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.285996 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:46.466693 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.466603 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:46.859094 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.858992 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh"] Jul 21 14:10:46.859404 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.859391 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="965e6fb7-39e7-4a96-b887-df812112adbb" containerName="main" Jul 21 14:10:46.859478 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.859405 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="965e6fb7-39e7-4a96-b887-df812112adbb" containerName="main" Jul 21 14:10:46.859478 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.859421 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="861275a7-188c-40eb-8d3c-ffe3c351b394" containerName="main" Jul 21 14:10:46.859478 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.859430 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="861275a7-188c-40eb-8d3c-ffe3c351b394" containerName="main" Jul 21 14:10:46.859713 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.859501 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="861275a7-188c-40eb-8d3c-ffe3c351b394" containerName="main" Jul 21 14:10:46.859713 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.859512 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="965e6fb7-39e7-4a96-b887-df812112adbb" containerName="main" Jul 21 14:10:46.862768 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.862748 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:46.867734 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.867710 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-47dbec60-openshift-default-dockercfg-2nqtd\"" Jul 21 14:10:46.907564 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.907528 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh"] Jul 21 14:10:46.916104 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:46.916049 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:47.013184 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.013140 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-workload-certs\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.013392 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.013202 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25879967-66fb-4361-912c-f37fdf3a6b56-istiod-ca-cert\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.013392 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.013233 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pqlb\" (UniqueName: \"kubernetes.io/projected/25879967-66fb-4361-912c-f37fdf3a6b56-kube-api-access-8pqlb\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.013392 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.013283 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25879967-66fb-4361-912c-f37fdf3a6b56-istio-podinfo\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.013392 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.013311 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-istio-data\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.013392 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.013363 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25879967-66fb-4361-912c-f37fdf3a6b56-istio-token\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.013613 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.013397 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-credential-socket\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.013613 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.013435 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-workload-socket\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.013613 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.013534 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-istio-envoy\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.114563 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.114451 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-workload-certs\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.114563 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.114505 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25879967-66fb-4361-912c-f37fdf3a6b56-istiod-ca-cert\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.114563 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.114530 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pqlb\" (UniqueName: \"kubernetes.io/projected/25879967-66fb-4361-912c-f37fdf3a6b56-kube-api-access-8pqlb\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.114917 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.114584 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25879967-66fb-4361-912c-f37fdf3a6b56-istio-podinfo\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.114917 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.114612 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-istio-data\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.114917 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.114667 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25879967-66fb-4361-912c-f37fdf3a6b56-istio-token\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.114917 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.114745 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-credential-socket\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.114917 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.114782 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-workload-socket\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.114917 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.114812 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-istio-envoy\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.115274 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.114970 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-workload-certs\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.115274 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.115253 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-istio-data\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.115380 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.115305 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-credential-socket\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.115380 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.115349 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25879967-66fb-4361-912c-f37fdf3a6b56-istiod-ca-cert\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.115606 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.115578 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-workload-socket\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.117183 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.117147 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-istio-envoy\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.117407 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.117390 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25879967-66fb-4361-912c-f37fdf3a6b56-istio-podinfo\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.124334 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.124299 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25879967-66fb-4361-912c-f37fdf3a6b56-istio-token\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.124467 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.124423 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pqlb\" (UniqueName: \"kubernetes.io/projected/25879967-66fb-4361-912c-f37fdf3a6b56-kube-api-access-8pqlb\") pod \"e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.175409 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.175365 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:47.354758 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.354731 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh"] Jul 21 14:10:47.357280 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:10:47.357234 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25879967_66fb_4361_912c_f37fdf3a6b56.slice/crio-b9e6e5dfc83085facb4be9984cb1b254bbead796086cd00848d6b410c513d733 WatchSource:0}: Error finding container b9e6e5dfc83085facb4be9984cb1b254bbead796086cd00848d6b410c513d733: Status 404 returned error can't find the container with id b9e6e5dfc83085facb4be9984cb1b254bbead796086cd00848d6b410c513d733 Jul 21 14:10:47.359723 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.359678 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:10:47.359812 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.359770 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:10:47.359812 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.359801 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:10:47.607859 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.607820 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" event={"ID":"25879967-66fb-4361-912c-f37fdf3a6b56","Type":"ContainerStarted","Data":"17f09b9828073e468e0d543c6ecf5d1d03440cac50ba74bbcce8d2f62d1cb6b2"} Jul 21 14:10:47.608052 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.607885 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" event={"ID":"25879967-66fb-4361-912c-f37fdf3a6b56","Type":"ContainerStarted","Data":"b9e6e5dfc83085facb4be9984cb1b254bbead796086cd00848d6b410c513d733"} Jul 21 14:10:47.637134 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.636995 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" podStartSLOduration=1.6369704550000002 podStartE2EDuration="1.636970455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:10:47.634145921 +0000 UTC m=+1608.785987032" watchObservedRunningTime="2026-07-21 14:10:47.636970455 +0000 UTC m=+1608.788811668" Jul 21 14:10:47.831559 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:47.831515 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:48.176494 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:48.176452 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:48.181462 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:48.181434 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:48.615303 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:48.615194 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:48.616274 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:48.616252 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:10:49.002534 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:49.002491 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:49.104344 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:49.104297 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:49.196258 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:49.196221 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:49.315235 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:49.315140 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:49.710246 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:49.710201 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:10:49.810922 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:10:49.810879 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:04.656951 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:04.656896 2567 patch_prober.go:28] interesting pod/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 14:11:04.657331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:04.657004 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" podUID="25879967-66fb-4361-912c-f37fdf3a6b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 14:11:06.718351 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:06.718315 2567 generic.go:356] "Generic (PLEG): container finished" podID="25879967-66fb-4361-912c-f37fdf3a6b56" containerID="17f09b9828073e468e0d543c6ecf5d1d03440cac50ba74bbcce8d2f62d1cb6b2" exitCode=0 Jul 21 14:11:06.718800 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:06.718391 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" event={"ID":"25879967-66fb-4361-912c-f37fdf3a6b56","Type":"ContainerDied","Data":"17f09b9828073e468e0d543c6ecf5d1d03440cac50ba74bbcce8d2f62d1cb6b2"} Jul 21 14:11:06.718903 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:06.718871 2567 scope.go:117] "RemoveContainer" containerID="17f09b9828073e468e0d543c6ecf5d1d03440cac50ba74bbcce8d2f62d1cb6b2" Jul 21 14:11:06.719889 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:06.719863 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:11:06.719945 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:06.719932 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:11:06.719983 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:06.719966 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:11:07.730666 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:07.730619 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" event={"ID":"25879967-66fb-4361-912c-f37fdf3a6b56","Type":"ContainerStarted","Data":"ad56466c9f70627812ee92d2f3a0e14bc85f7fef4b1ba1eb3cb1babe7d964c96"} Jul 21 14:11:08.175496 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:08.175465 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:11:08.175828 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:08.175805 2567 patch_prober.go:28] interesting pod/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 21 14:11:08.175887 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:08.175854 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" podUID="25879967-66fb-4361-912c-f37fdf3a6b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 21 14:11:09.176465 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:09.176425 2567 patch_prober.go:28] interesting pod/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 21 14:11:09.176926 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:09.176483 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" podUID="25879967-66fb-4361-912c-f37fdf3a6b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 21 14:11:10.175997 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:10.175949 2567 patch_prober.go:28] interesting pod/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 21 14:11:10.176309 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:10.176035 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" podUID="25879967-66fb-4361-912c-f37fdf3a6b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 21 14:11:11.175722 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:11.175668 2567 patch_prober.go:28] interesting pod/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 21 14:11:11.176115 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:11.175756 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" podUID="25879967-66fb-4361-912c-f37fdf3a6b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 21 14:11:12.183175 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:12.183114 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:11:12.183876 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:12.183739 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:11:12.185496 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:12.185456 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:11:25.198035 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:25.197997 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:29.668620 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:29.668578 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:29.806471 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:29.806433 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh"] Jul 21 14:11:29.806914 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:29.806659 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" podUID="25879967-66fb-4361-912c-f37fdf3a6b56" containerName="istio-proxy" containerID="cri-o://ad56466c9f70627812ee92d2f3a0e14bc85f7fef4b1ba1eb3cb1babe7d964c96" gracePeriod=30 Jul 21 14:11:29.834972 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:29.834924 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:30.092164 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.092064 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:30.338989 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.338948 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:30.430579 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.430542 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:30.560867 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.560831 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:30.864914 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.864831 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n"] Jul 21 14:11:30.868200 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.868182 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:30.872142 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.872121 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-1ebf4d7f-gw-openshift-default-dockercfg-gkmvv\"" Jul 21 14:11:30.890820 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.890789 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n"] Jul 21 14:11:30.902146 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.902113 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:30.932978 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.932943 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-workload-socket\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:30.933147 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.932995 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-podinfo\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:30.933147 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.933042 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-workload-certs\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:30.933147 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.933090 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-token\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:30.933147 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.933111 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsn5b\" (UniqueName: \"kubernetes.io/projected/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-kube-api-access-dsn5b\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:30.933147 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.933142 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-envoy\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:30.933342 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.933160 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-credential-socket\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:30.933342 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.933229 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-data\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:30.933342 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:30.933301 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istiod-ca-cert\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.034096 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.034055 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-envoy\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.034096 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.034102 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-credential-socket\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.034351 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.034217 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-data\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.034351 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.034281 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istiod-ca-cert\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.034351 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.034348 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-workload-socket\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.034523 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.034409 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-podinfo\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.034523 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.034440 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-workload-certs\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.034523 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.034496 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-token\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.034681 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.034523 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsn5b\" (UniqueName: \"kubernetes.io/projected/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-kube-api-access-dsn5b\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.034681 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.034596 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-credential-socket\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.034681 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.034661 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-data\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.034952 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.034931 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-workload-certs\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.035102 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.035077 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istiod-ca-cert\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.035201 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.035179 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-workload-socket\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.036587 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.036563 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-envoy\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.037108 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.037087 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-podinfo\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.044329 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.044303 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-token\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.044926 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.044910 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsn5b\" (UniqueName: \"kubernetes.io/projected/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-kube-api-access-dsn5b\") pod \"e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.116807 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.116716 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:31.178633 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.178599 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:31.326033 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.325999 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n"] Jul 21 14:11:31.326462 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:11:31.326430 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f9d5433_b7b0_4dbd_a632_b8f5966ee8ee.slice/crio-a570c600eec77b9ffab73b7f5b65d227f3094401dba1483f59fade309a3f0a53 WatchSource:0}: Error finding container a570c600eec77b9ffab73b7f5b65d227f3094401dba1483f59fade309a3f0a53: Status 404 returned error can't find the container with id a570c600eec77b9ffab73b7f5b65d227f3094401dba1483f59fade309a3f0a53 Jul 21 14:11:31.328957 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.328922 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:11:31.329052 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.328992 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:11:31.329052 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.329022 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:11:31.872335 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.872296 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" event={"ID":"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee","Type":"ContainerStarted","Data":"43cdaa345024581cb77eda9d50cadead637cddae13d1c40e5cc5539f336749cc"} Jul 21 14:11:31.872775 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:31.872353 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" event={"ID":"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee","Type":"ContainerStarted","Data":"a570c600eec77b9ffab73b7f5b65d227f3094401dba1483f59fade309a3f0a53"} Jul 21 14:11:32.179024 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:32.178981 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:32.184066 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:32.184036 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:32.208668 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:32.208613 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" podStartSLOduration=2.20859471 podStartE2EDuration="2.20859471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:11:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:11:31.915217359 +0000 UTC m=+1653.067058468" watchObservedRunningTime="2026-07-21 14:11:32.20859471 +0000 UTC m=+1653.360435818" Jul 21 14:11:32.884555 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:32.884524 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:32.885558 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:32.885539 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:33.400503 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:33.400466 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:33.516683 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:33.516650 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:33.584567 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:33.584531 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:34.033047 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:34.033014 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:34.236569 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:34.236539 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:11:35.066911 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.066842 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:11:35.182349 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182312 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-workload-socket\") pod \"25879967-66fb-4361-912c-f37fdf3a6b56\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " Jul 21 14:11:35.182349 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182355 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25879967-66fb-4361-912c-f37fdf3a6b56-istiod-ca-cert\") pod \"25879967-66fb-4361-912c-f37fdf3a6b56\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " Jul 21 14:11:35.182599 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182378 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25879967-66fb-4361-912c-f37fdf3a6b56-istio-token\") pod \"25879967-66fb-4361-912c-f37fdf3a6b56\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " Jul 21 14:11:35.182599 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182439 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-workload-certs\") pod \"25879967-66fb-4361-912c-f37fdf3a6b56\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " Jul 21 14:11:35.182599 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182461 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pqlb\" (UniqueName: \"kubernetes.io/projected/25879967-66fb-4361-912c-f37fdf3a6b56-kube-api-access-8pqlb\") pod \"25879967-66fb-4361-912c-f37fdf3a6b56\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " Jul 21 14:11:35.182599 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182476 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25879967-66fb-4361-912c-f37fdf3a6b56-istio-podinfo\") pod \"25879967-66fb-4361-912c-f37fdf3a6b56\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " Jul 21 14:11:35.182599 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182531 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-istio-envoy\") pod \"25879967-66fb-4361-912c-f37fdf3a6b56\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " Jul 21 14:11:35.182599 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182565 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-istio-data\") pod \"25879967-66fb-4361-912c-f37fdf3a6b56\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " Jul 21 14:11:35.182599 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182603 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-credential-socket\") pod \"25879967-66fb-4361-912c-f37fdf3a6b56\" (UID: \"25879967-66fb-4361-912c-f37fdf3a6b56\") " Jul 21 14:11:35.182964 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182675 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "25879967-66fb-4361-912c-f37fdf3a6b56" (UID: "25879967-66fb-4361-912c-f37fdf3a6b56"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:11:35.182964 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182835 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25879967-66fb-4361-912c-f37fdf3a6b56-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "25879967-66fb-4361-912c-f37fdf3a6b56" (UID: "25879967-66fb-4361-912c-f37fdf3a6b56"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 14:11:35.182964 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182842 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "25879967-66fb-4361-912c-f37fdf3a6b56" (UID: "25879967-66fb-4361-912c-f37fdf3a6b56"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:11:35.182964 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182957 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-workload-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:11:35.183177 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182978 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25879967-66fb-4361-912c-f37fdf3a6b56-istiod-ca-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:11:35.183177 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.182993 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-workload-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:11:35.183177 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.183079 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "25879967-66fb-4361-912c-f37fdf3a6b56" (UID: "25879967-66fb-4361-912c-f37fdf3a6b56"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:11:35.183756 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.183731 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-istio-data" (OuterVolumeSpecName: "istio-data") pod "25879967-66fb-4361-912c-f37fdf3a6b56" (UID: "25879967-66fb-4361-912c-f37fdf3a6b56"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:11:35.185045 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.185012 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25879967-66fb-4361-912c-f37fdf3a6b56-kube-api-access-8pqlb" (OuterVolumeSpecName: "kube-api-access-8pqlb") pod "25879967-66fb-4361-912c-f37fdf3a6b56" (UID: "25879967-66fb-4361-912c-f37fdf3a6b56"). InnerVolumeSpecName "kube-api-access-8pqlb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:11:35.185154 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.185054 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25879967-66fb-4361-912c-f37fdf3a6b56-istio-token" (OuterVolumeSpecName: "istio-token") pod "25879967-66fb-4361-912c-f37fdf3a6b56" (UID: "25879967-66fb-4361-912c-f37fdf3a6b56"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:11:35.185200 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.185172 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/25879967-66fb-4361-912c-f37fdf3a6b56-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "25879967-66fb-4361-912c-f37fdf3a6b56" (UID: "25879967-66fb-4361-912c-f37fdf3a6b56"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 14:11:35.185303 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.185268 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "25879967-66fb-4361-912c-f37fdf3a6b56" (UID: "25879967-66fb-4361-912c-f37fdf3a6b56"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:11:35.283466 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.283430 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-istio-envoy\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:11:35.283466 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.283461 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-istio-data\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:11:35.283466 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.283470 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25879967-66fb-4361-912c-f37fdf3a6b56-credential-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:11:35.283466 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.283480 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25879967-66fb-4361-912c-f37fdf3a6b56-istio-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:11:35.283772 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.283490 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pqlb\" (UniqueName: \"kubernetes.io/projected/25879967-66fb-4361-912c-f37fdf3a6b56-kube-api-access-8pqlb\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:11:35.283772 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.283500 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25879967-66fb-4361-912c-f37fdf3a6b56-istio-podinfo\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:11:35.903195 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.903159 2567 generic.go:356] "Generic (PLEG): container finished" podID="25879967-66fb-4361-912c-f37fdf3a6b56" containerID="ad56466c9f70627812ee92d2f3a0e14bc85f7fef4b1ba1eb3cb1babe7d964c96" exitCode=0 Jul 21 14:11:35.903366 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.903206 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" Jul 21 14:11:35.903366 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.903198 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" event={"ID":"25879967-66fb-4361-912c-f37fdf3a6b56","Type":"ContainerDied","Data":"ad56466c9f70627812ee92d2f3a0e14bc85f7fef4b1ba1eb3cb1babe7d964c96"} Jul 21 14:11:35.903366 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.903326 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh" event={"ID":"25879967-66fb-4361-912c-f37fdf3a6b56","Type":"ContainerDied","Data":"b9e6e5dfc83085facb4be9984cb1b254bbead796086cd00848d6b410c513d733"} Jul 21 14:11:35.903366 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.903356 2567 scope.go:117] "RemoveContainer" containerID="ad56466c9f70627812ee92d2f3a0e14bc85f7fef4b1ba1eb3cb1babe7d964c96" Jul 21 14:11:35.914721 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.914687 2567 scope.go:117] "RemoveContainer" containerID="17f09b9828073e468e0d543c6ecf5d1d03440cac50ba74bbcce8d2f62d1cb6b2" Jul 21 14:11:35.923746 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.923723 2567 scope.go:117] "RemoveContainer" containerID="ad56466c9f70627812ee92d2f3a0e14bc85f7fef4b1ba1eb3cb1babe7d964c96" Jul 21 14:11:35.924040 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:11:35.924019 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad56466c9f70627812ee92d2f3a0e14bc85f7fef4b1ba1eb3cb1babe7d964c96\": container with ID starting with ad56466c9f70627812ee92d2f3a0e14bc85f7fef4b1ba1eb3cb1babe7d964c96 not found: ID does not exist" containerID="ad56466c9f70627812ee92d2f3a0e14bc85f7fef4b1ba1eb3cb1babe7d964c96" Jul 21 14:11:35.924117 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.924052 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad56466c9f70627812ee92d2f3a0e14bc85f7fef4b1ba1eb3cb1babe7d964c96"} err="failed to get container status \"ad56466c9f70627812ee92d2f3a0e14bc85f7fef4b1ba1eb3cb1babe7d964c96\": rpc error: code = NotFound desc = could not find container \"ad56466c9f70627812ee92d2f3a0e14bc85f7fef4b1ba1eb3cb1babe7d964c96\": container with ID starting with ad56466c9f70627812ee92d2f3a0e14bc85f7fef4b1ba1eb3cb1babe7d964c96 not found: ID does not exist" Jul 21 14:11:35.924117 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.924072 2567 scope.go:117] "RemoveContainer" containerID="17f09b9828073e468e0d543c6ecf5d1d03440cac50ba74bbcce8d2f62d1cb6b2" Jul 21 14:11:35.924352 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:11:35.924327 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f09b9828073e468e0d543c6ecf5d1d03440cac50ba74bbcce8d2f62d1cb6b2\": container with ID starting with 17f09b9828073e468e0d543c6ecf5d1d03440cac50ba74bbcce8d2f62d1cb6b2 not found: ID does not exist" containerID="17f09b9828073e468e0d543c6ecf5d1d03440cac50ba74bbcce8d2f62d1cb6b2" Jul 21 14:11:35.924402 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.924358 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f09b9828073e468e0d543c6ecf5d1d03440cac50ba74bbcce8d2f62d1cb6b2"} err="failed to get container status \"17f09b9828073e468e0d543c6ecf5d1d03440cac50ba74bbcce8d2f62d1cb6b2\": rpc error: code = NotFound desc = could not find container \"17f09b9828073e468e0d543c6ecf5d1d03440cac50ba74bbcce8d2f62d1cb6b2\": container with ID starting with 17f09b9828073e468e0d543c6ecf5d1d03440cac50ba74bbcce8d2f62d1cb6b2 not found: ID does not exist" Jul 21 14:11:35.924740 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.924721 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh"] Jul 21 14:11:35.929130 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:35.929102 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-47dbec60-openshift-default-56fd55c6b9-zzfdh"] Jul 21 14:11:37.533785 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:37.533755 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25879967-66fb-4361-912c-f37fdf3a6b56" path="/var/lib/kubelet/pods/25879967-66fb-4361-912c-f37fdf3a6b56/volumes" Jul 21 14:11:48.035014 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:48.033728 2567 patch_prober.go:28] interesting pod/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": read tcp 10.132.0.2:36338->10.132.0.59:15021: read: connection reset by peer" start-of-body= Jul 21 14:11:48.035014 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:48.033818 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" podUID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": read tcp 10.132.0.2:36338->10.132.0.59:15021: read: connection reset by peer" Jul 21 14:11:48.988622 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:48.988592 2567 generic.go:356] "Generic (PLEG): container finished" podID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerID="43cdaa345024581cb77eda9d50cadead637cddae13d1c40e5cc5539f336749cc" exitCode=0 Jul 21 14:11:48.988842 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:48.988674 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" event={"ID":"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee","Type":"ContainerDied","Data":"43cdaa345024581cb77eda9d50cadead637cddae13d1c40e5cc5539f336749cc"} Jul 21 14:11:48.989194 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:48.989174 2567 scope.go:117] "RemoveContainer" containerID="43cdaa345024581cb77eda9d50cadead637cddae13d1c40e5cc5539f336749cc" Jul 21 14:11:48.990270 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:48.990241 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:11:48.990326 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:48.990314 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:11:48.990372 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:48.990347 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:11:49.996247 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:49.996214 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" event={"ID":"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee","Type":"ContainerStarted","Data":"44298a64d22139ae6b26b3b00cf58a2f34c5130b6f0b3e088325642f253af70f"} Jul 21 14:11:50.179179 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:50.179138 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:50.179492 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:50.179467 2567 patch_prober.go:28] interesting pod/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 21 14:11:50.179556 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:50.179520 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" podUID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 21 14:11:51.180170 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:51.180123 2567 patch_prober.go:28] interesting pod/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 21 14:11:51.180571 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:51.180200 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" podUID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 21 14:11:52.179060 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:52.179024 2567 patch_prober.go:28] interesting pod/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 21 14:11:52.179244 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:52.179097 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" podUID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 21 14:11:53.179808 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:53.179762 2567 patch_prober.go:28] interesting pod/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 21 14:11:53.180225 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:53.179838 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" podUID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 21 14:11:54.183484 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:54.183452 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:54.183953 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:54.183832 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:11:54.184506 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:11:54.184485 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:12:09.100226 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:09.100189 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:13.523579 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:13.523531 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:13.660800 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:13.660756 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n"] Jul 21 14:12:13.661059 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:13.661028 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" podUID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerName="istio-proxy" containerID="cri-o://44298a64d22139ae6b26b3b00cf58a2f34c5130b6f0b3e088325642f253af70f" gracePeriod=30 Jul 21 14:12:13.693536 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:13.693498 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:13.912908 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:13.912870 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:14.136283 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.136246 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:14.445392 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.445352 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5"] Jul 21 14:12:14.445964 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.445948 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25879967-66fb-4361-912c-f37fdf3a6b56" containerName="istio-proxy" Jul 21 14:12:14.446028 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.445968 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="25879967-66fb-4361-912c-f37fdf3a6b56" containerName="istio-proxy" Jul 21 14:12:14.446028 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.445984 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25879967-66fb-4361-912c-f37fdf3a6b56" containerName="istio-proxy" Jul 21 14:12:14.446028 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.445994 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="25879967-66fb-4361-912c-f37fdf3a6b56" containerName="istio-proxy" Jul 21 14:12:14.446131 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.446100 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="25879967-66fb-4361-912c-f37fdf3a6b56" containerName="istio-proxy" Jul 21 14:12:14.446131 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.446115 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="25879967-66fb-4361-912c-f37fdf3a6b56" containerName="istio-proxy" Jul 21 14:12:14.449722 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.449686 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.452407 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.452384 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-462af97f-openshift-default-dockercfg-8tzls\"" Jul 21 14:12:14.459510 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.459475 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5"] Jul 21 14:12:14.509317 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.509269 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:14.558160 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.558120 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-workload-socket\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.558160 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.558161 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-workload-certs\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.558621 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.558188 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-podinfo\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.558621 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.558211 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c1e6be-9a42-45ba-927c-b01b83c9b128-istiod-ca-cert\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.558621 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.558269 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-credential-socket\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.558621 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.558334 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-data\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.558621 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.558367 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tdll\" (UniqueName: \"kubernetes.io/projected/52c1e6be-9a42-45ba-927c-b01b83c9b128-kube-api-access-4tdll\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.558621 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.558410 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-token\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.558621 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.558430 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-envoy\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.659889 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.659843 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-workload-socket\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.660102 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.659899 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-workload-certs\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.660102 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.659938 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-podinfo\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.660102 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.659958 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c1e6be-9a42-45ba-927c-b01b83c9b128-istiod-ca-cert\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.660102 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.659990 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-credential-socket\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.660102 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.660067 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-data\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.660384 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.660310 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tdll\" (UniqueName: \"kubernetes.io/projected/52c1e6be-9a42-45ba-927c-b01b83c9b128-kube-api-access-4tdll\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.660384 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.660342 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-workload-socket\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.660496 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.660385 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-token\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.660496 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.660421 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-envoy\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.660594 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.660540 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-data\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.660594 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.660548 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-credential-socket\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.660730 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.660686 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-workload-certs\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.660811 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.660758 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c1e6be-9a42-45ba-927c-b01b83c9b128-istiod-ca-cert\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.662475 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.662454 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-envoy\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.662575 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.662475 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-podinfo\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.673158 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.673118 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tdll\" (UniqueName: \"kubernetes.io/projected/52c1e6be-9a42-45ba-927c-b01b83c9b128-kube-api-access-4tdll\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.673158 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.673123 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-token\") pod \"e2e-ait-462af97f-openshift-default-8648d86f7-nths5\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.712082 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.711982 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:14.762361 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.762314 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:14.919779 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.919750 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5"] Jul 21 14:12:14.920602 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:12:14.920575 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52c1e6be_9a42_45ba_927c_b01b83c9b128.slice/crio-64bfbab247d87dbab797bd91c391966bd28c9d706bb2c8c193289cbd04e93344 WatchSource:0}: Error finding container 64bfbab247d87dbab797bd91c391966bd28c9d706bb2c8c193289cbd04e93344: Status 404 returned error can't find the container with id 64bfbab247d87dbab797bd91c391966bd28c9d706bb2c8c193289cbd04e93344 Jul 21 14:12:14.922657 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.922626 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:12:14.922750 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.922691 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:12:14.922750 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:14.922739 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:12:15.146527 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:15.146486 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" event={"ID":"52c1e6be-9a42-45ba-927c-b01b83c9b128","Type":"ContainerStarted","Data":"bf3a5a65957ed3a069d4b73234fcf56afae4f717055ea4ef9387d18ccb63c44e"} Jul 21 14:12:15.146774 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:15.146542 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" event={"ID":"52c1e6be-9a42-45ba-927c-b01b83c9b128","Type":"ContainerStarted","Data":"64bfbab247d87dbab797bd91c391966bd28c9d706bb2c8c193289cbd04e93344"} Jul 21 14:12:15.169828 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:15.169758 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" podStartSLOduration=1.169734333 podStartE2EDuration="1.169734333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:12:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:12:15.165728407 +0000 UTC m=+1696.317569517" watchObservedRunningTime="2026-07-21 14:12:15.169734333 +0000 UTC m=+1696.321575443" Jul 21 14:12:15.763059 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:15.763023 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:15.768275 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:15.768247 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:16.152140 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:16.152107 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:16.153464 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:16.153441 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:16.912080 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:16.912042 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:17.017047 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:17.017004 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:17.221536 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:17.221444 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:17.321816 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:17.321775 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:17.418149 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:17.418106 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:18.933512 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:18.933442 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:12:19.103389 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.103350 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-token\") pod \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " Jul 21 14:12:19.103576 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.103439 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-podinfo\") pod \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " Jul 21 14:12:19.103576 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.103462 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-workload-socket\") pod \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " Jul 21 14:12:19.103576 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.103479 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-workload-certs\") pod \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " Jul 21 14:12:19.103576 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.103494 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsn5b\" (UniqueName: \"kubernetes.io/projected/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-kube-api-access-dsn5b\") pod \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " Jul 21 14:12:19.103576 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.103514 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istiod-ca-cert\") pod \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " Jul 21 14:12:19.103576 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.103557 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-envoy\") pod \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " Jul 21 14:12:19.103938 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.103587 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-data\") pod \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " Jul 21 14:12:19.103938 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.103634 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-credential-socket\") pod \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\" (UID: \"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee\") " Jul 21 14:12:19.103938 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.103805 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" (UID: "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:12:19.104095 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.104018 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" (UID: "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:12:19.104095 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.104055 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-workload-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:12:19.104095 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.104083 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" (UID: "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:12:19.104323 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.104290 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" (UID: "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 14:12:19.104680 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.104652 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-data" (OuterVolumeSpecName: "istio-data") pod "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" (UID: "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:12:19.105974 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.105931 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-kube-api-access-dsn5b" (OuterVolumeSpecName: "kube-api-access-dsn5b") pod "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" (UID: "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee"). InnerVolumeSpecName "kube-api-access-dsn5b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:12:19.106104 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.106035 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" (UID: "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:12:19.106196 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.106173 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-token" (OuterVolumeSpecName: "istio-token") pod "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" (UID: "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:12:19.106360 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.106331 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" (UID: "9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 14:12:19.170216 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.170181 2567 generic.go:356] "Generic (PLEG): container finished" podID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerID="44298a64d22139ae6b26b3b00cf58a2f34c5130b6f0b3e088325642f253af70f" exitCode=0 Jul 21 14:12:19.170418 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.170235 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" Jul 21 14:12:19.170418 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.170270 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" event={"ID":"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee","Type":"ContainerDied","Data":"44298a64d22139ae6b26b3b00cf58a2f34c5130b6f0b3e088325642f253af70f"} Jul 21 14:12:19.170418 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.170324 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n" event={"ID":"9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee","Type":"ContainerDied","Data":"a570c600eec77b9ffab73b7f5b65d227f3094401dba1483f59fade309a3f0a53"} Jul 21 14:12:19.170418 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.170353 2567 scope.go:117] "RemoveContainer" containerID="44298a64d22139ae6b26b3b00cf58a2f34c5130b6f0b3e088325642f253af70f" Jul 21 14:12:19.180748 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.180729 2567 scope.go:117] "RemoveContainer" containerID="43cdaa345024581cb77eda9d50cadead637cddae13d1c40e5cc5539f336749cc" Jul 21 14:12:19.189834 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.189809 2567 scope.go:117] "RemoveContainer" containerID="44298a64d22139ae6b26b3b00cf58a2f34c5130b6f0b3e088325642f253af70f" Jul 21 14:12:19.190113 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:12:19.190097 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44298a64d22139ae6b26b3b00cf58a2f34c5130b6f0b3e088325642f253af70f\": container with ID starting with 44298a64d22139ae6b26b3b00cf58a2f34c5130b6f0b3e088325642f253af70f not found: ID does not exist" containerID="44298a64d22139ae6b26b3b00cf58a2f34c5130b6f0b3e088325642f253af70f" Jul 21 14:12:19.190152 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.190123 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44298a64d22139ae6b26b3b00cf58a2f34c5130b6f0b3e088325642f253af70f"} err="failed to get container status \"44298a64d22139ae6b26b3b00cf58a2f34c5130b6f0b3e088325642f253af70f\": rpc error: code = NotFound desc = could not find container \"44298a64d22139ae6b26b3b00cf58a2f34c5130b6f0b3e088325642f253af70f\": container with ID starting with 44298a64d22139ae6b26b3b00cf58a2f34c5130b6f0b3e088325642f253af70f not found: ID does not exist" Jul 21 14:12:19.190152 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.190140 2567 scope.go:117] "RemoveContainer" containerID="43cdaa345024581cb77eda9d50cadead637cddae13d1c40e5cc5539f336749cc" Jul 21 14:12:19.190357 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:12:19.190339 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43cdaa345024581cb77eda9d50cadead637cddae13d1c40e5cc5539f336749cc\": container with ID starting with 43cdaa345024581cb77eda9d50cadead637cddae13d1c40e5cc5539f336749cc not found: ID does not exist" containerID="43cdaa345024581cb77eda9d50cadead637cddae13d1c40e5cc5539f336749cc" Jul 21 14:12:19.190396 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.190366 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43cdaa345024581cb77eda9d50cadead637cddae13d1c40e5cc5539f336749cc"} err="failed to get container status \"43cdaa345024581cb77eda9d50cadead637cddae13d1c40e5cc5539f336749cc\": rpc error: code = NotFound desc = could not find container \"43cdaa345024581cb77eda9d50cadead637cddae13d1c40e5cc5539f336749cc\": container with ID starting with 43cdaa345024581cb77eda9d50cadead637cddae13d1c40e5cc5539f336749cc not found: ID does not exist" Jul 21 14:12:19.195156 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.195130 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n"] Jul 21 14:12:19.199326 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.199303 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-1ebf4d7f-gw-openshift-default-744bc98cdc-56r5n"] Jul 21 14:12:19.205433 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.205408 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-podinfo\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:12:19.205433 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.205432 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-workload-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:12:19.205606 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.205444 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsn5b\" (UniqueName: \"kubernetes.io/projected/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-kube-api-access-dsn5b\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:12:19.205606 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.205453 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istiod-ca-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:12:19.205606 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.205462 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-envoy\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:12:19.205606 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.205470 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-data\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:12:19.205606 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.205478 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-credential-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:12:19.205606 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.205487 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee-istio-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:12:19.533923 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:19.533823 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" path="/var/lib/kubelet/pods/9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee/volumes" Jul 21 14:12:32.193928 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:32.193881 2567 patch_prober.go:28] interesting pod/e2e-ait-462af97f-openshift-default-8648d86f7-nths5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 14:12:32.196337 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:32.193961 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" podUID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 14:12:32.266153 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:32.266122 2567 generic.go:356] "Generic (PLEG): container finished" podID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerID="bf3a5a65957ed3a069d4b73234fcf56afae4f717055ea4ef9387d18ccb63c44e" exitCode=0 Jul 21 14:12:32.266299 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:32.266195 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" event={"ID":"52c1e6be-9a42-45ba-927c-b01b83c9b128","Type":"ContainerDied","Data":"bf3a5a65957ed3a069d4b73234fcf56afae4f717055ea4ef9387d18ccb63c44e"} Jul 21 14:12:32.266615 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:32.266600 2567 scope.go:117] "RemoveContainer" containerID="bf3a5a65957ed3a069d4b73234fcf56afae4f717055ea4ef9387d18ccb63c44e" Jul 21 14:12:32.267727 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:32.267675 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:12:32.267880 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:32.267786 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:12:32.267880 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:32.267820 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:12:33.273553 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:33.273513 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" event={"ID":"52c1e6be-9a42-45ba-927c-b01b83c9b128","Type":"ContainerStarted","Data":"d36334ff034f093118ead4a58b91b89c60532de886787c81df41d355496898ff"} Jul 21 14:12:33.762979 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:33.762934 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:33.763223 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:33.763196 2567 patch_prober.go:28] interesting pod/e2e-ait-462af97f-openshift-default-8648d86f7-nths5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 21 14:12:33.763274 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:33.763251 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" podUID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 21 14:12:34.762790 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:34.762749 2567 patch_prober.go:28] interesting pod/e2e-ait-462af97f-openshift-default-8648d86f7-nths5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 21 14:12:34.763218 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:34.762811 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" podUID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 21 14:12:35.763176 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:35.763128 2567 patch_prober.go:28] interesting pod/e2e-ait-462af97f-openshift-default-8648d86f7-nths5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 21 14:12:35.763745 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:35.763221 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" podUID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 21 14:12:36.763174 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:36.763137 2567 patch_prober.go:28] interesting pod/e2e-ait-462af97f-openshift-default-8648d86f7-nths5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 21 14:12:36.763364 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:36.763203 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" podUID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 21 14:12:37.766291 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:37.766256 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:37.766745 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:37.766546 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:37.767331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:37.767315 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:12:52.833405 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:52.833361 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:57.477280 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:57.477223 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:58.646909 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:58.646861 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5"] Jul 21 14:12:58.647407 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:58.647119 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" podUID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerName="istio-proxy" containerID="cri-o://d36334ff034f093118ead4a58b91b89c60532de886787c81df41d355496898ff" gracePeriod=30 Jul 21 14:12:58.669622 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:58.669584 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:58.894153 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:58.894115 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:59.113601 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.113562 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:59.393098 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.393003 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh"] Jul 21 14:12:59.393509 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.393487 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerName="istio-proxy" Jul 21 14:12:59.393597 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.393514 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerName="istio-proxy" Jul 21 14:12:59.393597 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.393528 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerName="istio-proxy" Jul 21 14:12:59.393597 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.393536 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerName="istio-proxy" Jul 21 14:12:59.393784 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.393679 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerName="istio-proxy" Jul 21 14:12:59.393784 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.393693 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f9d5433-b7b0-4dbd-a632-b8f5966ee8ee" containerName="istio-proxy" Jul 21 14:12:59.397256 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.397230 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.400401 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.400372 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-caef500b-openshift-default-dockercfg-q8rdg\"" Jul 21 14:12:59.410205 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.410169 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh"] Jul 21 14:12:59.416726 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.416672 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:59.481624 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.481577 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z265f\" (UniqueName: \"kubernetes.io/projected/f3053373-ae9d-42c9-81af-6dcd6c671ebb-kube-api-access-z265f\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.481624 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.481630 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-workload-certs\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.481948 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.481663 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-podinfo\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.481948 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.481686 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-token\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.481948 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.481782 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-envoy\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.481948 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.481830 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-data\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.481948 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.481891 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-workload-socket\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.482229 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.481956 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-credential-socket\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.482229 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.481989 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istiod-ca-cert\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.583193 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.583150 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-credential-socket\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.583334 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.583212 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istiod-ca-cert\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.583334 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.583276 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z265f\" (UniqueName: \"kubernetes.io/projected/f3053373-ae9d-42c9-81af-6dcd6c671ebb-kube-api-access-z265f\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.583419 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.583329 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-workload-certs\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.583419 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.583366 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-podinfo\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.583419 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.583395 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-token\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.583591 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.583498 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-envoy\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.583591 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.583546 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-data\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.583710 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.583604 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-workload-socket\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.583710 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.583652 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-credential-socket\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.583829 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.583693 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-workload-certs\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.584018 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.583997 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-workload-socket\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.584124 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.584101 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-data\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.584258 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.584234 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istiod-ca-cert\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.586169 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.586106 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-envoy\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.586298 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.586189 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-podinfo\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.593413 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.593375 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-token\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.593585 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.593389 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z265f\" (UniqueName: \"kubernetes.io/projected/f3053373-ae9d-42c9-81af-6dcd6c671ebb-kube-api-access-z265f\") pod \"e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.637575 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.637529 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:12:59.710292 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.710250 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:12:59.881766 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.881725 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh"] Jul 21 14:12:59.882653 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:12:59.882620 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3053373_ae9d_42c9_81af_6dcd6c671ebb.slice/crio-10d3a84d7768a6b09b050fc14364ea3bcdc6e6e369944c09e7a8d4f5b447b74b WatchSource:0}: Error finding container 10d3a84d7768a6b09b050fc14364ea3bcdc6e6e369944c09e7a8d4f5b447b74b: Status 404 returned error can't find the container with id 10d3a84d7768a6b09b050fc14364ea3bcdc6e6e369944c09e7a8d4f5b447b74b Jul 21 14:12:59.884959 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.884925 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:12:59.885078 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.885000 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:12:59.885078 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:12:59.885034 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:13:00.438294 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:00.438205 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" event={"ID":"f3053373-ae9d-42c9-81af-6dcd6c671ebb","Type":"ContainerStarted","Data":"3912a46790d6486591183dc980c1beeb35fc6f794122e2cca56b14519b2304c3"} Jul 21 14:13:00.438796 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:00.438351 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" event={"ID":"f3053373-ae9d-42c9-81af-6dcd6c671ebb","Type":"ContainerStarted","Data":"10d3a84d7768a6b09b050fc14364ea3bcdc6e6e369944c09e7a8d4f5b447b74b"} Jul 21 14:13:00.469511 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:00.469449 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" podStartSLOduration=1.469429654 podStartE2EDuration="1.469429654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:13:00.466341284 +0000 UTC m=+1741.618182397" watchObservedRunningTime="2026-07-21 14:13:00.469429654 +0000 UTC m=+1741.621270766" Jul 21 14:13:00.710474 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:00.710361 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:13:00.716125 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:00.716093 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:13:01.444487 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:01.444451 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:13:01.445507 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:01.445490 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:13:01.815603 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:01.815485 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:01.915612 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:01.915567 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:02.061136 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:02.061084 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:02.260618 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:02.260579 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:02.722116 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:02.722076 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:02.816622 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:02.816574 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:03.901292 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:03.901263 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:13:04.032645 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.032535 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-podinfo\") pod \"52c1e6be-9a42-45ba-927c-b01b83c9b128\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " Jul 21 14:13:04.032645 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.032598 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-data\") pod \"52c1e6be-9a42-45ba-927c-b01b83c9b128\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " Jul 21 14:13:04.032645 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.032615 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-token\") pod \"52c1e6be-9a42-45ba-927c-b01b83c9b128\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " Jul 21 14:13:04.032951 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.032764 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-workload-socket\") pod \"52c1e6be-9a42-45ba-927c-b01b83c9b128\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " Jul 21 14:13:04.032951 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.032815 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-credential-socket\") pod \"52c1e6be-9a42-45ba-927c-b01b83c9b128\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " Jul 21 14:13:04.032951 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.032845 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tdll\" (UniqueName: \"kubernetes.io/projected/52c1e6be-9a42-45ba-927c-b01b83c9b128-kube-api-access-4tdll\") pod \"52c1e6be-9a42-45ba-927c-b01b83c9b128\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " Jul 21 14:13:04.032951 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.032900 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-workload-certs\") pod \"52c1e6be-9a42-45ba-927c-b01b83c9b128\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " Jul 21 14:13:04.033165 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.032955 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c1e6be-9a42-45ba-927c-b01b83c9b128-istiod-ca-cert\") pod \"52c1e6be-9a42-45ba-927c-b01b83c9b128\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " Jul 21 14:13:04.033165 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.032982 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-envoy\") pod \"52c1e6be-9a42-45ba-927c-b01b83c9b128\" (UID: \"52c1e6be-9a42-45ba-927c-b01b83c9b128\") " Jul 21 14:13:04.033165 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.033006 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "52c1e6be-9a42-45ba-927c-b01b83c9b128" (UID: "52c1e6be-9a42-45ba-927c-b01b83c9b128"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:13:04.033329 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.033140 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "52c1e6be-9a42-45ba-927c-b01b83c9b128" (UID: "52c1e6be-9a42-45ba-927c-b01b83c9b128"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:13:04.033329 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.033234 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "52c1e6be-9a42-45ba-927c-b01b83c9b128" (UID: "52c1e6be-9a42-45ba-927c-b01b83c9b128"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:13:04.033443 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.033367 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-workload-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:04.033443 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.033386 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-workload-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:04.033443 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.033402 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-credential-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:04.033610 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.033473 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-data" (OuterVolumeSpecName: "istio-data") pod "52c1e6be-9a42-45ba-927c-b01b83c9b128" (UID: "52c1e6be-9a42-45ba-927c-b01b83c9b128"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:13:04.033610 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.033582 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52c1e6be-9a42-45ba-927c-b01b83c9b128-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "52c1e6be-9a42-45ba-927c-b01b83c9b128" (UID: "52c1e6be-9a42-45ba-927c-b01b83c9b128"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 14:13:04.035038 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.035015 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-token" (OuterVolumeSpecName: "istio-token") pod "52c1e6be-9a42-45ba-927c-b01b83c9b128" (UID: "52c1e6be-9a42-45ba-927c-b01b83c9b128"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:13:04.035185 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.035164 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "52c1e6be-9a42-45ba-927c-b01b83c9b128" (UID: "52c1e6be-9a42-45ba-927c-b01b83c9b128"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:13:04.035341 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.035314 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "52c1e6be-9a42-45ba-927c-b01b83c9b128" (UID: "52c1e6be-9a42-45ba-927c-b01b83c9b128"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 14:13:04.035454 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.035336 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c1e6be-9a42-45ba-927c-b01b83c9b128-kube-api-access-4tdll" (OuterVolumeSpecName: "kube-api-access-4tdll") pod "52c1e6be-9a42-45ba-927c-b01b83c9b128" (UID: "52c1e6be-9a42-45ba-927c-b01b83c9b128"). InnerVolumeSpecName "kube-api-access-4tdll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:13:04.134422 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.134366 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c1e6be-9a42-45ba-927c-b01b83c9b128-istiod-ca-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:04.134422 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.134410 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-envoy\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:04.134422 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.134419 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-podinfo\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:04.134422 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.134427 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-data\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:04.134422 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.134435 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c1e6be-9a42-45ba-927c-b01b83c9b128-istio-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:04.134763 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.134443 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tdll\" (UniqueName: \"kubernetes.io/projected/52c1e6be-9a42-45ba-927c-b01b83c9b128-kube-api-access-4tdll\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:04.463381 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.463343 2567 generic.go:356] "Generic (PLEG): container finished" podID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerID="d36334ff034f093118ead4a58b91b89c60532de886787c81df41d355496898ff" exitCode=0 Jul 21 14:13:04.463618 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.463403 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" Jul 21 14:13:04.463618 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.463416 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" event={"ID":"52c1e6be-9a42-45ba-927c-b01b83c9b128","Type":"ContainerDied","Data":"d36334ff034f093118ead4a58b91b89c60532de886787c81df41d355496898ff"} Jul 21 14:13:04.463618 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.463478 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5" event={"ID":"52c1e6be-9a42-45ba-927c-b01b83c9b128","Type":"ContainerDied","Data":"64bfbab247d87dbab797bd91c391966bd28c9d706bb2c8c193289cbd04e93344"} Jul 21 14:13:04.463618 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.463501 2567 scope.go:117] "RemoveContainer" containerID="d36334ff034f093118ead4a58b91b89c60532de886787c81df41d355496898ff" Jul 21 14:13:04.473721 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.473684 2567 scope.go:117] "RemoveContainer" containerID="bf3a5a65957ed3a069d4b73234fcf56afae4f717055ea4ef9387d18ccb63c44e" Jul 21 14:13:04.482360 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.482338 2567 scope.go:117] "RemoveContainer" containerID="d36334ff034f093118ead4a58b91b89c60532de886787c81df41d355496898ff" Jul 21 14:13:04.482657 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:13:04.482635 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d36334ff034f093118ead4a58b91b89c60532de886787c81df41d355496898ff\": container with ID starting with d36334ff034f093118ead4a58b91b89c60532de886787c81df41d355496898ff not found: ID does not exist" containerID="d36334ff034f093118ead4a58b91b89c60532de886787c81df41d355496898ff" Jul 21 14:13:04.482759 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.482669 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d36334ff034f093118ead4a58b91b89c60532de886787c81df41d355496898ff"} err="failed to get container status \"d36334ff034f093118ead4a58b91b89c60532de886787c81df41d355496898ff\": rpc error: code = NotFound desc = could not find container \"d36334ff034f093118ead4a58b91b89c60532de886787c81df41d355496898ff\": container with ID starting with d36334ff034f093118ead4a58b91b89c60532de886787c81df41d355496898ff not found: ID does not exist" Jul 21 14:13:04.482844 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.482829 2567 scope.go:117] "RemoveContainer" containerID="bf3a5a65957ed3a069d4b73234fcf56afae4f717055ea4ef9387d18ccb63c44e" Jul 21 14:13:04.483082 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:13:04.483063 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf3a5a65957ed3a069d4b73234fcf56afae4f717055ea4ef9387d18ccb63c44e\": container with ID starting with bf3a5a65957ed3a069d4b73234fcf56afae4f717055ea4ef9387d18ccb63c44e not found: ID does not exist" containerID="bf3a5a65957ed3a069d4b73234fcf56afae4f717055ea4ef9387d18ccb63c44e" Jul 21 14:13:04.483181 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.483086 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf3a5a65957ed3a069d4b73234fcf56afae4f717055ea4ef9387d18ccb63c44e"} err="failed to get container status \"bf3a5a65957ed3a069d4b73234fcf56afae4f717055ea4ef9387d18ccb63c44e\": rpc error: code = NotFound desc = could not find container \"bf3a5a65957ed3a069d4b73234fcf56afae4f717055ea4ef9387d18ccb63c44e\": container with ID starting with bf3a5a65957ed3a069d4b73234fcf56afae4f717055ea4ef9387d18ccb63c44e not found: ID does not exist" Jul 21 14:13:04.488716 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.488648 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5"] Jul 21 14:13:04.492263 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:04.492230 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-462af97f-openshift-default-8648d86f7-nths5"] Jul 21 14:13:05.533671 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:05.533632 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52c1e6be-9a42-45ba-927c-b01b83c9b128" path="/var/lib/kubelet/pods/52c1e6be-9a42-45ba-927c-b01b83c9b128/volumes" Jul 21 14:13:16.446643 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:16.446600 2567 patch_prober.go:28] interesting pod/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 21 14:13:16.447161 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:16.446669 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" podUID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 21 14:13:16.541076 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:16.540985 2567 generic.go:356] "Generic (PLEG): container finished" podID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerID="3912a46790d6486591183dc980c1beeb35fc6f794122e2cca56b14519b2304c3" exitCode=0 Jul 21 14:13:16.541076 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:16.541060 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" event={"ID":"f3053373-ae9d-42c9-81af-6dcd6c671ebb","Type":"ContainerDied","Data":"3912a46790d6486591183dc980c1beeb35fc6f794122e2cca56b14519b2304c3"} Jul 21 14:13:16.541522 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:16.541507 2567 scope.go:117] "RemoveContainer" containerID="3912a46790d6486591183dc980c1beeb35fc6f794122e2cca56b14519b2304c3" Jul 21 14:13:16.542693 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:16.542662 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:13:16.542796 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:16.542779 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:13:16.542843 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:16.542823 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:13:17.549110 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:17.549071 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" event={"ID":"f3053373-ae9d-42c9-81af-6dcd6c671ebb","Type":"ContainerStarted","Data":"4a69a7301643f5d697e7cfd732b91a7cc19a56263990ae8ea2768f0522d59822"} Jul 21 14:13:17.711840 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:17.711228 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:13:17.711840 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:17.711417 2567 patch_prober.go:28] interesting pod/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 21 14:13:17.711840 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:17.711502 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" podUID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 21 14:13:18.711342 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:18.711292 2567 patch_prober.go:28] interesting pod/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 21 14:13:18.711918 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:18.711369 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" podUID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 21 14:13:19.711193 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:19.711152 2567 patch_prober.go:28] interesting pod/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 21 14:13:19.711414 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:19.711226 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" podUID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 21 14:13:20.711077 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:20.711022 2567 patch_prober.go:28] interesting pod/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 21 14:13:20.711312 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:20.711100 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" podUID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 21 14:13:21.714476 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:21.714433 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:13:21.714908 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:21.714810 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:13:21.715579 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:21.715558 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:13:28.143542 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:28.143503 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:28.243961 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:28.243925 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:28.699891 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:28.699852 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:28.844466 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:28.844433 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:30.652725 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:30.652661 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:35.694629 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:35.694581 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:37.302374 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:37.302329 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:41.805140 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:41.805097 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:41.926043 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:41.926000 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh"] Jul 21 14:13:41.926285 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:41.926261 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" podUID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerName="istio-proxy" containerID="cri-o://4a69a7301643f5d697e7cfd732b91a7cc19a56263990ae8ea2768f0522d59822" gracePeriod=30 Jul 21 14:13:41.954181 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:41.954142 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:42.305922 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:42.305884 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:42.964212 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:42.964168 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:43.061444 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:43.061403 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:46.276521 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:46.276481 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:47.199791 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.199727 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:13:47.358664 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.358625 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-envoy\") pod \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " Jul 21 14:13:47.358664 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.358664 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istiod-ca-cert\") pod \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " Jul 21 14:13:47.359231 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.358683 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-credential-socket\") pod \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " Jul 21 14:13:47.359231 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.358725 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-workload-socket\") pod \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " Jul 21 14:13:47.359231 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.358767 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z265f\" (UniqueName: \"kubernetes.io/projected/f3053373-ae9d-42c9-81af-6dcd6c671ebb-kube-api-access-z265f\") pod \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " Jul 21 14:13:47.359231 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.358798 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-podinfo\") pod \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " Jul 21 14:13:47.359231 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.358820 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-data\") pod \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " Jul 21 14:13:47.359231 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.358854 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-workload-certs\") pod \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " Jul 21 14:13:47.359231 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.358979 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-token\") pod \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\" (UID: \"f3053373-ae9d-42c9-81af-6dcd6c671ebb\") " Jul 21 14:13:47.359231 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.358974 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f3053373-ae9d-42c9-81af-6dcd6c671ebb" (UID: "f3053373-ae9d-42c9-81af-6dcd6c671ebb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:13:47.359231 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.359117 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f3053373-ae9d-42c9-81af-6dcd6c671ebb" (UID: "f3053373-ae9d-42c9-81af-6dcd6c671ebb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 14:13:47.359732 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.359345 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istiod-ca-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:47.359732 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.359365 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-credential-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:47.359732 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.359539 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f3053373-ae9d-42c9-81af-6dcd6c671ebb" (UID: "f3053373-ae9d-42c9-81af-6dcd6c671ebb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:13:47.359732 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.359649 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f3053373-ae9d-42c9-81af-6dcd6c671ebb" (UID: "f3053373-ae9d-42c9-81af-6dcd6c671ebb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:13:47.359970 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.359946 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-data" (OuterVolumeSpecName: "istio-data") pod "f3053373-ae9d-42c9-81af-6dcd6c671ebb" (UID: "f3053373-ae9d-42c9-81af-6dcd6c671ebb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:13:47.361197 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.361163 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3053373-ae9d-42c9-81af-6dcd6c671ebb-kube-api-access-z265f" (OuterVolumeSpecName: "kube-api-access-z265f") pod "f3053373-ae9d-42c9-81af-6dcd6c671ebb" (UID: "f3053373-ae9d-42c9-81af-6dcd6c671ebb"). InnerVolumeSpecName "kube-api-access-z265f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:13:47.361197 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.361183 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-token" (OuterVolumeSpecName: "istio-token") pod "f3053373-ae9d-42c9-81af-6dcd6c671ebb" (UID: "f3053373-ae9d-42c9-81af-6dcd6c671ebb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:13:47.361365 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.361196 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f3053373-ae9d-42c9-81af-6dcd6c671ebb" (UID: "f3053373-ae9d-42c9-81af-6dcd6c671ebb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:13:47.361676 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.361645 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f3053373-ae9d-42c9-81af-6dcd6c671ebb" (UID: "f3053373-ae9d-42c9-81af-6dcd6c671ebb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 14:13:47.460856 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.460626 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:47.460856 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.460667 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-envoy\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:47.460856 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.460681 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-workload-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:47.460856 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.460728 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z265f\" (UniqueName: \"kubernetes.io/projected/f3053373-ae9d-42c9-81af-6dcd6c671ebb-kube-api-access-z265f\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:47.460856 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.460744 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-podinfo\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:47.460856 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.460757 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-istio-data\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:47.460856 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.460772 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3053373-ae9d-42c9-81af-6dcd6c671ebb-workload-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:13:47.716410 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.716320 2567 generic.go:356] "Generic (PLEG): container finished" podID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerID="4a69a7301643f5d697e7cfd732b91a7cc19a56263990ae8ea2768f0522d59822" exitCode=0 Jul 21 14:13:47.716410 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.716365 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" event={"ID":"f3053373-ae9d-42c9-81af-6dcd6c671ebb","Type":"ContainerDied","Data":"4a69a7301643f5d697e7cfd732b91a7cc19a56263990ae8ea2768f0522d59822"} Jul 21 14:13:47.716597 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.716413 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" Jul 21 14:13:47.716597 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.716429 2567 scope.go:117] "RemoveContainer" containerID="4a69a7301643f5d697e7cfd732b91a7cc19a56263990ae8ea2768f0522d59822" Jul 21 14:13:47.716597 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.716417 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh" event={"ID":"f3053373-ae9d-42c9-81af-6dcd6c671ebb","Type":"ContainerDied","Data":"10d3a84d7768a6b09b050fc14364ea3bcdc6e6e369944c09e7a8d4f5b447b74b"} Jul 21 14:13:47.732741 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.732688 2567 scope.go:117] "RemoveContainer" containerID="3912a46790d6486591183dc980c1beeb35fc6f794122e2cca56b14519b2304c3" Jul 21 14:13:47.739073 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.739026 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh"] Jul 21 14:13:47.742646 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.742609 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-caef500b-openshift-default-787db5ddb7-mhjnh"] Jul 21 14:13:47.744724 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.744683 2567 scope.go:117] "RemoveContainer" containerID="4a69a7301643f5d697e7cfd732b91a7cc19a56263990ae8ea2768f0522d59822" Jul 21 14:13:47.745173 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:13:47.745146 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a69a7301643f5d697e7cfd732b91a7cc19a56263990ae8ea2768f0522d59822\": container with ID starting with 4a69a7301643f5d697e7cfd732b91a7cc19a56263990ae8ea2768f0522d59822 not found: ID does not exist" containerID="4a69a7301643f5d697e7cfd732b91a7cc19a56263990ae8ea2768f0522d59822" Jul 21 14:13:47.745268 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.745187 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a69a7301643f5d697e7cfd732b91a7cc19a56263990ae8ea2768f0522d59822"} err="failed to get container status \"4a69a7301643f5d697e7cfd732b91a7cc19a56263990ae8ea2768f0522d59822\": rpc error: code = NotFound desc = could not find container \"4a69a7301643f5d697e7cfd732b91a7cc19a56263990ae8ea2768f0522d59822\": container with ID starting with 4a69a7301643f5d697e7cfd732b91a7cc19a56263990ae8ea2768f0522d59822 not found: ID does not exist" Jul 21 14:13:47.745268 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.745215 2567 scope.go:117] "RemoveContainer" containerID="3912a46790d6486591183dc980c1beeb35fc6f794122e2cca56b14519b2304c3" Jul 21 14:13:47.745570 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:13:47.745539 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3912a46790d6486591183dc980c1beeb35fc6f794122e2cca56b14519b2304c3\": container with ID starting with 3912a46790d6486591183dc980c1beeb35fc6f794122e2cca56b14519b2304c3 not found: ID does not exist" containerID="3912a46790d6486591183dc980c1beeb35fc6f794122e2cca56b14519b2304c3" Jul 21 14:13:47.745570 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:47.745581 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3912a46790d6486591183dc980c1beeb35fc6f794122e2cca56b14519b2304c3"} err="failed to get container status \"3912a46790d6486591183dc980c1beeb35fc6f794122e2cca56b14519b2304c3\": rpc error: code = NotFound desc = could not find container \"3912a46790d6486591183dc980c1beeb35fc6f794122e2cca56b14519b2304c3\": container with ID starting with 3912a46790d6486591183dc980c1beeb35fc6f794122e2cca56b14519b2304c3 not found: ID does not exist" Jul 21 14:13:49.533127 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:49.533092 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" path="/var/lib/kubelet/pods/f3053373-ae9d-42c9-81af-6dcd6c671ebb/volumes" Jul 21 14:13:52.435224 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:52.435137 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:52.560123 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:52.560090 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:52.706498 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:52.706401 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:54.325535 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:54.325498 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:54.428829 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:54.428791 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:13:59.501444 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:59.501414 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:13:59.507293 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:13:59.507260 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:14:19.383058 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:14:19.383014 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:14:25.947160 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:14:25.947120 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:14:26.146582 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:14:26.146543 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:14:26.328851 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:14:26.328766 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:14:42.005331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:14:42.005292 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:14:48.520395 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:14:48.520351 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:14:48.595815 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:14:48.595771 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:14:48.698729 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:14:48.698659 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:14:48.800416 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:14:48.800257 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:14:59.284135 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:14:59.284095 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:00.167375 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:00.167328 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:07.472212 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:07.472152 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:07.583564 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:07.583510 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:07.705408 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:07.705359 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:11.029011 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:11.028966 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:11.126298 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:11.126240 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:16.938768 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:16.938676 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:17.031163 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:17.031116 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:17.654015 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:17.653966 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:23.792714 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:23.792599 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:23.899476 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:23.899418 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:24.093745 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.093532 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:24.359377 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.359208 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll"] Jul 21 14:15:24.359825 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.359807 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerName="istio-proxy" Jul 21 14:15:24.359923 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.359828 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerName="istio-proxy" Jul 21 14:15:24.359923 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.359852 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerName="istio-proxy" Jul 21 14:15:24.359923 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.359861 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerName="istio-proxy" Jul 21 14:15:24.359923 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.359878 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerName="istio-proxy" Jul 21 14:15:24.359923 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.359888 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerName="istio-proxy" Jul 21 14:15:24.359923 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.359906 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerName="istio-proxy" Jul 21 14:15:24.359923 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.359915 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerName="istio-proxy" Jul 21 14:15:24.360279 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.360006 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerName="istio-proxy" Jul 21 14:15:24.360279 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.360018 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerName="istio-proxy" Jul 21 14:15:24.360279 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.360029 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="52c1e6be-9a42-45ba-927c-b01b83c9b128" containerName="istio-proxy" Jul 21 14:15:24.363645 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.363607 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.367046 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.367013 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5b75b902-openshift-default-dockercfg-s44s6\"" Jul 21 14:15:24.380424 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.380359 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll"] Jul 21 14:15:24.383395 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.383332 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-envoy\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.383619 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.383429 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-podinfo\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.383619 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.383477 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-token\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.383619 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.383528 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-credential-socket\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.383619 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.383574 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnssj\" (UniqueName: \"kubernetes.io/projected/f5e79a8a-6273-4266-8358-5ba505c696ab-kube-api-access-lnssj\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.383809 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.383637 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-workload-socket\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.383809 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.383746 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-data\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.383809 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.383792 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-workload-certs\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.383944 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.383818 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5e79a8a-6273-4266-8358-5ba505c696ab-istiod-ca-cert\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.388657 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.388602 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:24.484936 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.484876 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-workload-socket\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.485167 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.484975 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-data\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.485167 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.485022 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-workload-certs\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.485167 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.485049 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5e79a8a-6273-4266-8358-5ba505c696ab-istiod-ca-cert\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.485167 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.485117 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-envoy\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.485167 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.485156 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-podinfo\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.485448 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.485189 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-token\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.485448 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.485225 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-credential-socket\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.485448 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.485269 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnssj\" (UniqueName: \"kubernetes.io/projected/f5e79a8a-6273-4266-8358-5ba505c696ab-kube-api-access-lnssj\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.485611 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.485445 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-workload-socket\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.485611 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.485469 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-data\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.486144 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.485924 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-workload-certs\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.486144 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.486096 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-credential-socket\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.486444 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.486422 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5e79a8a-6273-4266-8358-5ba505c696ab-istiod-ca-cert\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.488046 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.488011 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:24.488648 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.488624 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-envoy\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.489749 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.489726 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-podinfo\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.498768 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.498723 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnssj\" (UniqueName: \"kubernetes.io/projected/f5e79a8a-6273-4266-8358-5ba505c696ab-kube-api-access-lnssj\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.500527 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.500482 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-token\") pod \"e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.679489 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.679435 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:24.931861 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.931811 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll"] Jul 21 14:15:24.935071 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:15:24.935028 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5e79a8a_6273_4266_8358_5ba505c696ab.slice/crio-0b3efcebcfd81fcddc92c034c41d626d54abb5b92e1e9491cacb49640e56169f WatchSource:0}: Error finding container 0b3efcebcfd81fcddc92c034c41d626d54abb5b92e1e9491cacb49640e56169f: Status 404 returned error can't find the container with id 0b3efcebcfd81fcddc92c034c41d626d54abb5b92e1e9491cacb49640e56169f Jul 21 14:15:24.938326 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.938278 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:15:24.938525 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.938388 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:15:24.938525 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:24.938440 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:15:25.309274 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:25.309157 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" event={"ID":"f5e79a8a-6273-4266-8358-5ba505c696ab","Type":"ContainerStarted","Data":"bac33ae57106b43e3d3aea222f1371d81e2a9070feff916bf9c48584bfdd6782"} Jul 21 14:15:25.309559 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:25.309537 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" event={"ID":"f5e79a8a-6273-4266-8358-5ba505c696ab","Type":"ContainerStarted","Data":"0b3efcebcfd81fcddc92c034c41d626d54abb5b92e1e9491cacb49640e56169f"} Jul 21 14:15:25.336115 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:25.336028 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" podStartSLOduration=1.336004887 podStartE2EDuration="1.336004887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:15:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:15:25.33257539 +0000 UTC m=+1886.484416493" watchObservedRunningTime="2026-07-21 14:15:25.336004887 +0000 UTC m=+1886.487845996" Jul 21 14:15:25.680065 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:25.680025 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:26.686551 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:26.686505 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:27.323411 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:27.323354 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:27.324819 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:27.324778 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:27.335130 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:27.335076 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:27.446612 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:27.446567 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:27.541005 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:27.540964 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:27.918813 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:27.918765 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:28.166688 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:28.166632 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:28.268762 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:28.268610 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:31.805259 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:31.805215 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:35.515820 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:35.515759 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:35.579821 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:35.579775 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:39.349406 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:39.349358 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:39.450338 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:39.450295 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:41.420932 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:41.420896 2567 generic.go:356] "Generic (PLEG): container finished" podID="f5e79a8a-6273-4266-8358-5ba505c696ab" containerID="bac33ae57106b43e3d3aea222f1371d81e2a9070feff916bf9c48584bfdd6782" exitCode=0 Jul 21 14:15:41.421371 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:41.420948 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" event={"ID":"f5e79a8a-6273-4266-8358-5ba505c696ab","Type":"ContainerDied","Data":"bac33ae57106b43e3d3aea222f1371d81e2a9070feff916bf9c48584bfdd6782"} Jul 21 14:15:41.421427 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:41.421397 2567 scope.go:117] "RemoveContainer" containerID="bac33ae57106b43e3d3aea222f1371d81e2a9070feff916bf9c48584bfdd6782" Jul 21 14:15:41.422493 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:41.422461 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:15:41.422571 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:41.422536 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:15:41.422571 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:41.422568 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:15:42.323678 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:42.323617 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:42.433281 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:42.433231 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" event={"ID":"f5e79a8a-6273-4266-8358-5ba505c696ab","Type":"ContainerStarted","Data":"e9149f1ae9d4acae5ad634bd4366afa50eadfe826fccf481d7471f9a1381ed38"} Jul 21 14:15:42.680406 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:42.680363 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:42.681509 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:42.681475 2567 patch_prober.go:28] interesting pod/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 21 14:15:42.681641 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:42.681536 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" podUID="f5e79a8a-6273-4266-8358-5ba505c696ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 21 14:15:43.680770 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:43.680729 2567 patch_prober.go:28] interesting pod/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 21 14:15:43.681209 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:43.680791 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" podUID="f5e79a8a-6273-4266-8358-5ba505c696ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 21 14:15:44.680205 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:44.680162 2567 patch_prober.go:28] interesting pod/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 21 14:15:44.680402 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:44.680228 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" podUID="f5e79a8a-6273-4266-8358-5ba505c696ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 21 14:15:45.680676 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:45.680634 2567 patch_prober.go:28] interesting pod/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 21 14:15:45.681143 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:45.680729 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" podUID="f5e79a8a-6273-4266-8358-5ba505c696ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 21 14:15:46.685342 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:46.685311 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:46.685796 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:46.685594 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:46.686352 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:46.686336 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:15:50.294880 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:50.294827 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:54.816613 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:54.816578 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:15:55.187683 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:15:55.187608 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:01.389712 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:01.389665 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:01.496266 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:01.496224 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:01.715680 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:01.715637 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll"] Jul 21 14:16:01.715961 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:01.715924 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" podUID="f5e79a8a-6273-4266-8358-5ba505c696ab" containerName="istio-proxy" containerID="cri-o://e9149f1ae9d4acae5ad634bd4366afa50eadfe826fccf481d7471f9a1381ed38" gracePeriod=30 Jul 21 14:16:01.769379 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:01.769347 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:02.014057 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:02.013952 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:02.134162 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:02.134123 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:02.230439 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:02.230394 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:02.384196 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:02.384102 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:06.980979 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:06.980890 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:16:07.026256 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.026217 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-token\") pod \"f5e79a8a-6273-4266-8358-5ba505c696ab\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " Jul 21 14:16:07.026455 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.026264 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-workload-certs\") pod \"f5e79a8a-6273-4266-8358-5ba505c696ab\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " Jul 21 14:16:07.026455 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.026307 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-envoy\") pod \"f5e79a8a-6273-4266-8358-5ba505c696ab\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " Jul 21 14:16:07.026455 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.026345 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5e79a8a-6273-4266-8358-5ba505c696ab-istiod-ca-cert\") pod \"f5e79a8a-6273-4266-8358-5ba505c696ab\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " Jul 21 14:16:07.026455 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.026388 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-workload-socket\") pod \"f5e79a8a-6273-4266-8358-5ba505c696ab\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " Jul 21 14:16:07.026990 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.026459 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnssj\" (UniqueName: \"kubernetes.io/projected/f5e79a8a-6273-4266-8358-5ba505c696ab-kube-api-access-lnssj\") pod \"f5e79a8a-6273-4266-8358-5ba505c696ab\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " Jul 21 14:16:07.026990 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.026485 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-data\") pod \"f5e79a8a-6273-4266-8358-5ba505c696ab\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " Jul 21 14:16:07.026990 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.026648 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f5e79a8a-6273-4266-8358-5ba505c696ab" (UID: "f5e79a8a-6273-4266-8358-5ba505c696ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:16:07.026990 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.026742 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-credential-socket\") pod \"f5e79a8a-6273-4266-8358-5ba505c696ab\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " Jul 21 14:16:07.026990 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.026818 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-podinfo\") pod \"f5e79a8a-6273-4266-8358-5ba505c696ab\" (UID: \"f5e79a8a-6273-4266-8358-5ba505c696ab\") " Jul 21 14:16:07.026990 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.026916 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f5e79a8a-6273-4266-8358-5ba505c696ab" (UID: "f5e79a8a-6273-4266-8358-5ba505c696ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:16:07.027315 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.027023 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f5e79a8a-6273-4266-8358-5ba505c696ab" (UID: "f5e79a8a-6273-4266-8358-5ba505c696ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:16:07.027315 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.027071 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5e79a8a-6273-4266-8358-5ba505c696ab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f5e79a8a-6273-4266-8358-5ba505c696ab" (UID: "f5e79a8a-6273-4266-8358-5ba505c696ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 14:16:07.027315 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.027178 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-workload-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:16:07.027315 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.027196 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5e79a8a-6273-4266-8358-5ba505c696ab-istiod-ca-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:16:07.027315 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.027208 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-workload-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:16:07.027315 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.027221 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-credential-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:16:07.027792 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.027771 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-data" (OuterVolumeSpecName: "istio-data") pod "f5e79a8a-6273-4266-8358-5ba505c696ab" (UID: "f5e79a8a-6273-4266-8358-5ba505c696ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:16:07.028983 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.028952 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5e79a8a-6273-4266-8358-5ba505c696ab-kube-api-access-lnssj" (OuterVolumeSpecName: "kube-api-access-lnssj") pod "f5e79a8a-6273-4266-8358-5ba505c696ab" (UID: "f5e79a8a-6273-4266-8358-5ba505c696ab"). InnerVolumeSpecName "kube-api-access-lnssj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:16:07.029109 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.029091 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-token" (OuterVolumeSpecName: "istio-token") pod "f5e79a8a-6273-4266-8358-5ba505c696ab" (UID: "f5e79a8a-6273-4266-8358-5ba505c696ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:16:07.029173 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.029134 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f5e79a8a-6273-4266-8358-5ba505c696ab" (UID: "f5e79a8a-6273-4266-8358-5ba505c696ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:16:07.029356 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.029329 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f5e79a8a-6273-4266-8358-5ba505c696ab" (UID: "f5e79a8a-6273-4266-8358-5ba505c696ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 14:16:07.128671 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.128634 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnssj\" (UniqueName: \"kubernetes.io/projected/f5e79a8a-6273-4266-8358-5ba505c696ab-kube-api-access-lnssj\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:16:07.128671 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.128668 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-data\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:16:07.128671 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.128680 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-podinfo\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:16:07.128671 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.128688 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:16:07.128985 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.128727 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5e79a8a-6273-4266-8358-5ba505c696ab-istio-envoy\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:16:07.584020 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.583985 2567 generic.go:356] "Generic (PLEG): container finished" podID="f5e79a8a-6273-4266-8358-5ba505c696ab" containerID="e9149f1ae9d4acae5ad634bd4366afa50eadfe826fccf481d7471f9a1381ed38" exitCode=0 Jul 21 14:16:07.584218 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.584043 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" Jul 21 14:16:07.584218 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.584072 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" event={"ID":"f5e79a8a-6273-4266-8358-5ba505c696ab","Type":"ContainerDied","Data":"e9149f1ae9d4acae5ad634bd4366afa50eadfe826fccf481d7471f9a1381ed38"} Jul 21 14:16:07.584218 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.584143 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll" event={"ID":"f5e79a8a-6273-4266-8358-5ba505c696ab","Type":"ContainerDied","Data":"0b3efcebcfd81fcddc92c034c41d626d54abb5b92e1e9491cacb49640e56169f"} Jul 21 14:16:07.584218 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.584168 2567 scope.go:117] "RemoveContainer" containerID="e9149f1ae9d4acae5ad634bd4366afa50eadfe826fccf481d7471f9a1381ed38" Jul 21 14:16:07.597060 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.597031 2567 scope.go:117] "RemoveContainer" containerID="bac33ae57106b43e3d3aea222f1371d81e2a9070feff916bf9c48584bfdd6782" Jul 21 14:16:07.605447 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.605407 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll"] Jul 21 14:16:07.608841 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.608815 2567 scope.go:117] "RemoveContainer" containerID="e9149f1ae9d4acae5ad634bd4366afa50eadfe826fccf481d7471f9a1381ed38" Jul 21 14:16:07.609230 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:16:07.609205 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9149f1ae9d4acae5ad634bd4366afa50eadfe826fccf481d7471f9a1381ed38\": container with ID starting with e9149f1ae9d4acae5ad634bd4366afa50eadfe826fccf481d7471f9a1381ed38 not found: ID does not exist" containerID="e9149f1ae9d4acae5ad634bd4366afa50eadfe826fccf481d7471f9a1381ed38" Jul 21 14:16:07.609300 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.609242 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9149f1ae9d4acae5ad634bd4366afa50eadfe826fccf481d7471f9a1381ed38"} err="failed to get container status \"e9149f1ae9d4acae5ad634bd4366afa50eadfe826fccf481d7471f9a1381ed38\": rpc error: code = NotFound desc = could not find container \"e9149f1ae9d4acae5ad634bd4366afa50eadfe826fccf481d7471f9a1381ed38\": container with ID starting with e9149f1ae9d4acae5ad634bd4366afa50eadfe826fccf481d7471f9a1381ed38 not found: ID does not exist" Jul 21 14:16:07.609300 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.609263 2567 scope.go:117] "RemoveContainer" containerID="bac33ae57106b43e3d3aea222f1371d81e2a9070feff916bf9c48584bfdd6782" Jul 21 14:16:07.609589 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:16:07.609560 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bac33ae57106b43e3d3aea222f1371d81e2a9070feff916bf9c48584bfdd6782\": container with ID starting with bac33ae57106b43e3d3aea222f1371d81e2a9070feff916bf9c48584bfdd6782 not found: ID does not exist" containerID="bac33ae57106b43e3d3aea222f1371d81e2a9070feff916bf9c48584bfdd6782" Jul 21 14:16:07.609650 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.609590 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac33ae57106b43e3d3aea222f1371d81e2a9070feff916bf9c48584bfdd6782"} err="failed to get container status \"bac33ae57106b43e3d3aea222f1371d81e2a9070feff916bf9c48584bfdd6782\": rpc error: code = NotFound desc = could not find container \"bac33ae57106b43e3d3aea222f1371d81e2a9070feff916bf9c48584bfdd6782\": container with ID starting with bac33ae57106b43e3d3aea222f1371d81e2a9070feff916bf9c48584bfdd6782 not found: ID does not exist" Jul 21 14:16:07.610240 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:07.610220 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5b75b902-openshift-default-75d6b44f7f-qpgll"] Jul 21 14:16:09.152398 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:09.152355 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:09.222510 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:09.222466 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:09.327225 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:09.327179 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:09.421189 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:09.421092 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:09.520223 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:09.520175 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:09.534058 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:09.534017 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5e79a8a-6273-4266-8358-5ba505c696ab" path="/var/lib/kubelet/pods/f5e79a8a-6273-4266-8358-5ba505c696ab/volumes" Jul 21 14:16:15.494019 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:15.493978 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:15.593470 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:15.593428 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:22.294789 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.294751 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:22.593167 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.593068 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n"] Jul 21 14:16:22.593574 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.593553 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5e79a8a-6273-4266-8358-5ba505c696ab" containerName="istio-proxy" Jul 21 14:16:22.593671 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.593576 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5e79a8a-6273-4266-8358-5ba505c696ab" containerName="istio-proxy" Jul 21 14:16:22.593671 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.593603 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5e79a8a-6273-4266-8358-5ba505c696ab" containerName="istio-proxy" Jul 21 14:16:22.593671 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.593612 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5e79a8a-6273-4266-8358-5ba505c696ab" containerName="istio-proxy" Jul 21 14:16:22.593870 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.593736 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3053373-ae9d-42c9-81af-6dcd6c671ebb" containerName="istio-proxy" Jul 21 14:16:22.593870 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.593750 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5e79a8a-6273-4266-8358-5ba505c696ab" containerName="istio-proxy" Jul 21 14:16:22.593870 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.593768 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5e79a8a-6273-4266-8358-5ba505c696ab" containerName="istio-proxy" Jul 21 14:16:22.598574 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.598544 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.601543 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.601512 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9b3e9882-openshift-default-dockercfg-nz8qm\"" Jul 21 14:16:22.613423 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.613383 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n"] Jul 21 14:16:22.615459 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.615423 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:22.677504 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.677466 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-workload-socket\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.677722 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.677531 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-credential-socket\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.677722 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.677581 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-data\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.677722 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.677615 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-envoy\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.677722 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.677644 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d9gq\" (UniqueName: \"kubernetes.io/projected/0a96344c-aa36-4d7b-b3a4-18969af55c48-kube-api-access-4d9gq\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.677722 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.677668 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a96344c-aa36-4d7b-b3a4-18969af55c48-istiod-ca-cert\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.677722 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.677691 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-token\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.677980 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.677767 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-podinfo\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.677980 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.677810 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-workload-certs\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.722651 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.722611 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:22.779231 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.779192 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-workload-socket\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.779474 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.779249 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-credential-socket\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.779474 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.779282 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-data\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.779474 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.779306 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-envoy\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.779474 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.779344 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d9gq\" (UniqueName: \"kubernetes.io/projected/0a96344c-aa36-4d7b-b3a4-18969af55c48-kube-api-access-4d9gq\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.779474 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.779380 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a96344c-aa36-4d7b-b3a4-18969af55c48-istiod-ca-cert\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.779474 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.779417 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-token\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.779474 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.779458 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-podinfo\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.779901 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.779514 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-workload-certs\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.779901 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.779795 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-workload-socket\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.779901 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.779833 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-credential-socket\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.780059 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.779910 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-workload-certs\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.780059 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.779950 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-data\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.780550 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.780521 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a96344c-aa36-4d7b-b3a4-18969af55c48-istiod-ca-cert\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.782327 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.782295 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-podinfo\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.782476 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.782449 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-envoy\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.789377 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.789341 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-token\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.791432 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.791401 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d9gq\" (UniqueName: \"kubernetes.io/projected/0a96344c-aa36-4d7b-b3a4-18969af55c48-kube-api-access-4d9gq\") pod \"e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:22.910581 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:22.910539 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:23.072040 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:23.072005 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n"] Jul 21 14:16:23.074199 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:16:23.074162 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a96344c_aa36_4d7b_b3a4_18969af55c48.slice/crio-d08b9ad0eb087b35a89dd199ad24e08806f4e7022607a3814f6be9aa6dde9c3b WatchSource:0}: Error finding container d08b9ad0eb087b35a89dd199ad24e08806f4e7022607a3814f6be9aa6dde9c3b: Status 404 returned error can't find the container with id d08b9ad0eb087b35a89dd199ad24e08806f4e7022607a3814f6be9aa6dde9c3b Jul 21 14:16:23.076814 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:23.076762 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:16:23.076914 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:23.076871 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:16:23.076956 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:23.076914 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:16:23.682802 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:23.682751 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" event={"ID":"0a96344c-aa36-4d7b-b3a4-18969af55c48","Type":"ContainerStarted","Data":"3c9dd9e89d084651c09ecb08918e4ae21cc36928d29292c4f4fde1669a1d2636"} Jul 21 14:16:23.683422 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:23.682832 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" event={"ID":"0a96344c-aa36-4d7b-b3a4-18969af55c48","Type":"ContainerStarted","Data":"d08b9ad0eb087b35a89dd199ad24e08806f4e7022607a3814f6be9aa6dde9c3b"} Jul 21 14:16:23.705672 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:23.705608 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" podStartSLOduration=1.705586203 podStartE2EDuration="1.705586203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:16:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:16:23.703865068 +0000 UTC m=+1944.855706194" watchObservedRunningTime="2026-07-21 14:16:23.705586203 +0000 UTC m=+1944.857427311" Jul 21 14:16:23.910679 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:23.910633 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:24.917468 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:24.917433 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:25.693220 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:25.693167 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:25.696968 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:25.696937 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:25.698343 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:25.698297 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:25.778184 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:25.778133 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:25.863044 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:25.862990 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:26.113271 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:26.113158 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:26.513853 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:26.513814 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:26.614515 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:26.614476 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:30.207929 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:30.207885 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:37.578388 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:37.578336 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:16:39.790004 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:39.789960 2567 generic.go:356] "Generic (PLEG): container finished" podID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerID="3c9dd9e89d084651c09ecb08918e4ae21cc36928d29292c4f4fde1669a1d2636" exitCode=0 Jul 21 14:16:39.790462 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:39.790032 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" event={"ID":"0a96344c-aa36-4d7b-b3a4-18969af55c48","Type":"ContainerDied","Data":"3c9dd9e89d084651c09ecb08918e4ae21cc36928d29292c4f4fde1669a1d2636"} Jul 21 14:16:39.790508 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:39.790481 2567 scope.go:117] "RemoveContainer" containerID="3c9dd9e89d084651c09ecb08918e4ae21cc36928d29292c4f4fde1669a1d2636" Jul 21 14:16:39.791655 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:39.791621 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:16:39.791798 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:39.791729 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:16:39.791798 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:39.791785 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:16:40.697792 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:40.697736 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:40.798109 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:40.798065 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" event={"ID":"0a96344c-aa36-4d7b-b3a4-18969af55c48","Type":"ContainerStarted","Data":"c2e42342e186c20b3d1e8a838e9b9618c635e3e80a5acb4ffae8d3b981dd6bcf"} Jul 21 14:16:40.911357 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:40.911290 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:40.913103 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:40.913065 2567 patch_prober.go:28] interesting pod/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 21 14:16:40.913257 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:40.913144 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" podUID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 21 14:16:41.911690 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:41.911633 2567 patch_prober.go:28] interesting pod/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 21 14:16:41.912276 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:41.911740 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" podUID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 21 14:16:42.911475 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:42.911427 2567 patch_prober.go:28] interesting pod/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 21 14:16:42.912310 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:42.911808 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" podUID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 21 14:16:43.911980 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:43.911927 2567 patch_prober.go:28] interesting pod/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 21 14:16:43.912225 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:43.912016 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" podUID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 21 14:16:44.911531 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:44.911484 2567 patch_prober.go:28] interesting pod/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 21 14:16:44.912227 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:44.912187 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" podUID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 21 14:16:45.915783 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:45.915747 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:45.916275 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:45.916097 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:16:45.924654 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:16:45.924612 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:17:19.121554 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:17:19.121507 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:17:23.767423 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:17:23.767354 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:17:25.971790 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:17:25.971743 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:17:31.061111 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:17:31.061074 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:17:43.221501 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:17:43.221463 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:08.616604 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:08.616540 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:08.764586 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:08.764543 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:15.240243 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:15.240191 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:21.434049 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:21.434012 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:21.534310 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:21.534219 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:21.691570 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:21.691473 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n"] Jul 21 14:18:21.691765 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:21.691725 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" podUID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerName="istio-proxy" containerID="cri-o://c2e42342e186c20b3d1e8a838e9b9618c635e3e80a5acb4ffae8d3b981dd6bcf" gracePeriod=30 Jul 21 14:18:21.718538 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:21.718500 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:21.934373 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:21.934332 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:23.038853 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:23.038811 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:23.132753 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:23.132723 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:23.259267 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:23.259237 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:23.979857 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:23.979820 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:24.068262 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:24.068229 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:24.168966 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:24.168924 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:25.586347 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:25.586289 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:26.960742 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:26.960662 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:18:27.105465 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.105424 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d9gq\" (UniqueName: \"kubernetes.io/projected/0a96344c-aa36-4d7b-b3a4-18969af55c48-kube-api-access-4d9gq\") pod \"0a96344c-aa36-4d7b-b3a4-18969af55c48\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " Jul 21 14:18:27.105669 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.105474 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-data\") pod \"0a96344c-aa36-4d7b-b3a4-18969af55c48\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " Jul 21 14:18:27.105669 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.105492 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-token\") pod \"0a96344c-aa36-4d7b-b3a4-18969af55c48\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " Jul 21 14:18:27.105669 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.105514 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-podinfo\") pod \"0a96344c-aa36-4d7b-b3a4-18969af55c48\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " Jul 21 14:18:27.105669 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.105582 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a96344c-aa36-4d7b-b3a4-18969af55c48-istiod-ca-cert\") pod \"0a96344c-aa36-4d7b-b3a4-18969af55c48\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " Jul 21 14:18:27.105669 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.105613 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-workload-socket\") pod \"0a96344c-aa36-4d7b-b3a4-18969af55c48\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " Jul 21 14:18:27.105669 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.105643 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-workload-certs\") pod \"0a96344c-aa36-4d7b-b3a4-18969af55c48\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " Jul 21 14:18:27.106122 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.105760 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-credential-socket\") pod \"0a96344c-aa36-4d7b-b3a4-18969af55c48\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " Jul 21 14:18:27.106122 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.105832 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-envoy\") pod \"0a96344c-aa36-4d7b-b3a4-18969af55c48\" (UID: \"0a96344c-aa36-4d7b-b3a4-18969af55c48\") " Jul 21 14:18:27.106122 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.105980 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0a96344c-aa36-4d7b-b3a4-18969af55c48" (UID: "0a96344c-aa36-4d7b-b3a4-18969af55c48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:18:27.106122 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.106004 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0a96344c-aa36-4d7b-b3a4-18969af55c48" (UID: "0a96344c-aa36-4d7b-b3a4-18969af55c48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:18:27.106122 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.106012 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a96344c-aa36-4d7b-b3a4-18969af55c48-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0a96344c-aa36-4d7b-b3a4-18969af55c48" (UID: "0a96344c-aa36-4d7b-b3a4-18969af55c48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 14:18:27.106401 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.106168 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-workload-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:18:27.106401 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.106189 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a96344c-aa36-4d7b-b3a4-18969af55c48-istiod-ca-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:18:27.106401 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.106205 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-workload-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:18:27.106401 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.106202 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0a96344c-aa36-4d7b-b3a4-18969af55c48" (UID: "0a96344c-aa36-4d7b-b3a4-18969af55c48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:18:27.106811 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.106781 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-data" (OuterVolumeSpecName: "istio-data") pod "0a96344c-aa36-4d7b-b3a4-18969af55c48" (UID: "0a96344c-aa36-4d7b-b3a4-18969af55c48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:18:27.108286 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.108250 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-token" (OuterVolumeSpecName: "istio-token") pod "0a96344c-aa36-4d7b-b3a4-18969af55c48" (UID: "0a96344c-aa36-4d7b-b3a4-18969af55c48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:18:27.108454 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.108412 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0a96344c-aa36-4d7b-b3a4-18969af55c48" (UID: "0a96344c-aa36-4d7b-b3a4-18969af55c48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:18:27.108574 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.108524 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0a96344c-aa36-4d7b-b3a4-18969af55c48" (UID: "0a96344c-aa36-4d7b-b3a4-18969af55c48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 14:18:27.108574 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.108534 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a96344c-aa36-4d7b-b3a4-18969af55c48-kube-api-access-4d9gq" (OuterVolumeSpecName: "kube-api-access-4d9gq") pod "0a96344c-aa36-4d7b-b3a4-18969af55c48" (UID: "0a96344c-aa36-4d7b-b3a4-18969af55c48"). InnerVolumeSpecName "kube-api-access-4d9gq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:18:27.206891 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.206850 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-credential-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:18:27.206891 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.206883 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-envoy\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:18:27.206891 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.206895 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d9gq\" (UniqueName: \"kubernetes.io/projected/0a96344c-aa36-4d7b-b3a4-18969af55c48-kube-api-access-4d9gq\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:18:27.207167 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.206909 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-data\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:18:27.207167 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.206922 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:18:27.207167 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.206935 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a96344c-aa36-4d7b-b3a4-18969af55c48-istio-podinfo\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:18:27.442394 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.442353 2567 generic.go:356] "Generic (PLEG): container finished" podID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerID="c2e42342e186c20b3d1e8a838e9b9618c635e3e80a5acb4ffae8d3b981dd6bcf" exitCode=0 Jul 21 14:18:27.442606 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.442419 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" Jul 21 14:18:27.442606 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.442433 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" event={"ID":"0a96344c-aa36-4d7b-b3a4-18969af55c48","Type":"ContainerDied","Data":"c2e42342e186c20b3d1e8a838e9b9618c635e3e80a5acb4ffae8d3b981dd6bcf"} Jul 21 14:18:27.442606 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.442493 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n" event={"ID":"0a96344c-aa36-4d7b-b3a4-18969af55c48","Type":"ContainerDied","Data":"d08b9ad0eb087b35a89dd199ad24e08806f4e7022607a3814f6be9aa6dde9c3b"} Jul 21 14:18:27.442606 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.442526 2567 scope.go:117] "RemoveContainer" containerID="c2e42342e186c20b3d1e8a838e9b9618c635e3e80a5acb4ffae8d3b981dd6bcf" Jul 21 14:18:27.454548 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.454527 2567 scope.go:117] "RemoveContainer" containerID="3c9dd9e89d084651c09ecb08918e4ae21cc36928d29292c4f4fde1669a1d2636" Jul 21 14:18:27.464591 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.464565 2567 scope.go:117] "RemoveContainer" containerID="c2e42342e186c20b3d1e8a838e9b9618c635e3e80a5acb4ffae8d3b981dd6bcf" Jul 21 14:18:27.464968 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:18:27.464941 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2e42342e186c20b3d1e8a838e9b9618c635e3e80a5acb4ffae8d3b981dd6bcf\": container with ID starting with c2e42342e186c20b3d1e8a838e9b9618c635e3e80a5acb4ffae8d3b981dd6bcf not found: ID does not exist" containerID="c2e42342e186c20b3d1e8a838e9b9618c635e3e80a5acb4ffae8d3b981dd6bcf" Jul 21 14:18:27.465048 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.464983 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2e42342e186c20b3d1e8a838e9b9618c635e3e80a5acb4ffae8d3b981dd6bcf"} err="failed to get container status \"c2e42342e186c20b3d1e8a838e9b9618c635e3e80a5acb4ffae8d3b981dd6bcf\": rpc error: code = NotFound desc = could not find container \"c2e42342e186c20b3d1e8a838e9b9618c635e3e80a5acb4ffae8d3b981dd6bcf\": container with ID starting with c2e42342e186c20b3d1e8a838e9b9618c635e3e80a5acb4ffae8d3b981dd6bcf not found: ID does not exist" Jul 21 14:18:27.465048 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.465012 2567 scope.go:117] "RemoveContainer" containerID="3c9dd9e89d084651c09ecb08918e4ae21cc36928d29292c4f4fde1669a1d2636" Jul 21 14:18:27.465312 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:18:27.465291 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c9dd9e89d084651c09ecb08918e4ae21cc36928d29292c4f4fde1669a1d2636\": container with ID starting with 3c9dd9e89d084651c09ecb08918e4ae21cc36928d29292c4f4fde1669a1d2636 not found: ID does not exist" containerID="3c9dd9e89d084651c09ecb08918e4ae21cc36928d29292c4f4fde1669a1d2636" Jul 21 14:18:27.465371 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.465322 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c9dd9e89d084651c09ecb08918e4ae21cc36928d29292c4f4fde1669a1d2636"} err="failed to get container status \"3c9dd9e89d084651c09ecb08918e4ae21cc36928d29292c4f4fde1669a1d2636\": rpc error: code = NotFound desc = could not find container \"3c9dd9e89d084651c09ecb08918e4ae21cc36928d29292c4f4fde1669a1d2636\": container with ID starting with 3c9dd9e89d084651c09ecb08918e4ae21cc36928d29292c4f4fde1669a1d2636 not found: ID does not exist" Jul 21 14:18:27.472417 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.472387 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n"] Jul 21 14:18:27.479799 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.479765 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9b3e9882-openshift-default-5f5c6b6b67-cbq4n"] Jul 21 14:18:27.534636 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:27.534593 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a96344c-aa36-4d7b-b3a4-18969af55c48" path="/var/lib/kubelet/pods/0a96344c-aa36-4d7b-b3a4-18969af55c48/volumes" Jul 21 14:18:28.666316 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:28.666277 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:28.767611 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:28.767574 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:32.080113 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:32.080061 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:32.292953 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:32.292915 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:32.799302 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:32.799245 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:39.638171 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:39.638128 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:39.820438 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:39.820384 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:39.946709 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:39.946662 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:50.425548 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:50.425514 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:50.814379 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:50.814276 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:57.072164 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:57.072115 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:57.172616 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:57.172574 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:18:59.540000 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:59.539969 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:18:59.546359 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:18:59.546340 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:19:11.742824 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:11.742778 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:18.717061 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:18.717016 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:18.801170 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:18.801127 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:19.064632 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.064509 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:19.355857 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.355769 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh"] Jul 21 14:19:19.356159 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.356145 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerName="istio-proxy" Jul 21 14:19:19.356206 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.356161 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerName="istio-proxy" Jul 21 14:19:19.356241 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.356225 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerName="istio-proxy" Jul 21 14:19:19.356241 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.356235 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerName="istio-proxy" Jul 21 14:19:19.359416 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.359392 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.362570 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.362551 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-ce81f3-openshift-default-dockercfg-b4k9t\"" Jul 21 14:19:19.370982 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.370953 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh"] Jul 21 14:19:19.381421 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.381376 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:19.495300 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.495243 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-podinfo\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.495530 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.495312 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-workload-socket\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.495530 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.495352 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vscjb\" (UniqueName: \"kubernetes.io/projected/cd80f796-9f59-4499-9012-9c77b6f9befd-kube-api-access-vscjb\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.495530 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.495388 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-workload-certs\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.495530 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.495440 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-token\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.495530 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.495511 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd80f796-9f59-4499-9012-9c77b6f9befd-istiod-ca-cert\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.495813 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.495544 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-envoy\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.495813 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.495605 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-data\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.495813 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.495632 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-credential-socket\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.596299 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.596266 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-workload-socket\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.596299 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.596308 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vscjb\" (UniqueName: \"kubernetes.io/projected/cd80f796-9f59-4499-9012-9c77b6f9befd-kube-api-access-vscjb\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.596578 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.596329 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-workload-certs\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.596578 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.596452 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-token\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.596578 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.596537 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd80f796-9f59-4499-9012-9c77b6f9befd-istiod-ca-cert\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.596763 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.596583 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-envoy\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.596763 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.596615 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-data\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.596763 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.596646 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-credential-socket\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.596763 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.596758 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-podinfo\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.596981 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.596814 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-workload-certs\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.596981 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.596951 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-data\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.597091 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.597037 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-credential-socket\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.597242 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.597217 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-workload-socket\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.597365 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.597339 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd80f796-9f59-4499-9012-9c77b6f9befd-istiod-ca-cert\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.599004 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.598975 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-envoy\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.599148 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.599129 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-podinfo\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.606733 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.606640 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-token\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.607064 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.607040 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vscjb\" (UniqueName: \"kubernetes.io/projected/cd80f796-9f59-4499-9012-9c77b6f9befd-kube-api-access-vscjb\") pod \"e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.671294 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.671255 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:19.823597 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.823566 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh"] Jul 21 14:19:19.825636 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:19:19.825608 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd80f796_9f59_4499_9012_9c77b6f9befd.slice/crio-cc601fcff8c9490ddb42c429e488964b6b5f71a065c25454207b2156e6452eda WatchSource:0}: Error finding container cc601fcff8c9490ddb42c429e488964b6b5f71a065c25454207b2156e6452eda: Status 404 returned error can't find the container with id cc601fcff8c9490ddb42c429e488964b6b5f71a065c25454207b2156e6452eda Jul 21 14:19:19.827950 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.827904 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:19:19.828036 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.828000 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:19:19.828079 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:19.828045 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:19:20.052372 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:20.052330 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:20.754149 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:20.754104 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" event={"ID":"cd80f796-9f59-4499-9012-9c77b6f9befd","Type":"ContainerStarted","Data":"74f784462dc9973e4310e119211e655d6ab145eb2911bd0adc269f46e8e1f5f9"} Jul 21 14:19:20.754344 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:20.754207 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" event={"ID":"cd80f796-9f59-4499-9012-9c77b6f9befd","Type":"ContainerStarted","Data":"cc601fcff8c9490ddb42c429e488964b6b5f71a065c25454207b2156e6452eda"} Jul 21 14:19:20.777392 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:20.777335 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" podStartSLOduration=1.777318468 podStartE2EDuration="1.777318468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:19:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:19:20.774487285 +0000 UTC m=+2121.926328393" watchObservedRunningTime="2026-07-21 14:19:20.777318468 +0000 UTC m=+2121.929159577" Jul 21 14:19:21.672031 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:21.671984 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:21.677073 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:21.677043 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:21.760373 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:21.760346 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:21.761472 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:21.761449 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:22.702015 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:22.701978 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:22.908856 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:22.908825 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:23.099360 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:23.099252 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:23.249025 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:23.248980 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:25.831380 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:25.831326 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:25.931240 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:25.931204 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:27.184420 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.184379 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:27.511677 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.511580 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:27.803715 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.803620 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6"] Jul 21 14:19:27.804026 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.804013 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerName="istio-proxy" Jul 21 14:19:27.804069 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.804028 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a96344c-aa36-4d7b-b3a4-18969af55c48" containerName="istio-proxy" Jul 21 14:19:27.807522 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.807499 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.812077 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.812056 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-faeda6-openshift-default-dockercfg-n7vvb\"" Jul 21 14:19:27.830115 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.830084 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6"] Jul 21 14:19:27.877523 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.877489 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-workload-socket\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.877687 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.877526 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-data\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.877687 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.877552 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-token\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.877687 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.877605 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922388ac-5be3-4163-bd9f-0e0fe10575e2-istiod-ca-cert\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.877845 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.877734 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-podinfo\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.877845 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.877772 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-envoy\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.877845 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.877789 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v24k5\" (UniqueName: \"kubernetes.io/projected/922388ac-5be3-4163-bd9f-0e0fe10575e2-kube-api-access-v24k5\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.877958 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.877868 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-credential-socket\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.877958 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.877887 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-workload-certs\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.907035 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.907002 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:27.979156 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.979125 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-envoy\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.979156 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.979158 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v24k5\" (UniqueName: \"kubernetes.io/projected/922388ac-5be3-4163-bd9f-0e0fe10575e2-kube-api-access-v24k5\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.979371 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.979209 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-credential-socket\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.979371 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.979232 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-workload-certs\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.979371 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.979297 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-workload-socket\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.979371 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.979321 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-data\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.979371 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.979345 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-token\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.979604 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.979383 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922388ac-5be3-4163-bd9f-0e0fe10575e2-istiod-ca-cert\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.979738 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.979710 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-credential-socket\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.979814 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.979747 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-workload-socket\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.979901 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.979865 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-workload-certs\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.979901 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.979890 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-podinfo\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.980142 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.980125 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-data\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.980207 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.980195 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922388ac-5be3-4163-bd9f-0e0fe10575e2-istiod-ca-cert\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.981458 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.981440 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-envoy\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.982028 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.982009 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-podinfo\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.995664 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.995640 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-token\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:27.995867 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:27.995851 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v24k5\" (UniqueName: \"kubernetes.io/projected/922388ac-5be3-4163-bd9f-0e0fe10575e2-kube-api-access-v24k5\") pod \"e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:28.120152 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:28.120070 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:28.130295 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:28.130255 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:28.280740 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:28.280713 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6"] Jul 21 14:19:28.281838 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:19:28.281801 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922388ac_5be3_4163_bd9f_0e0fe10575e2.slice/crio-42dc52be9a9c33a66f774ecc61ecf8785b9b65918aacc0a9a86b4d387966e81d WatchSource:0}: Error finding container 42dc52be9a9c33a66f774ecc61ecf8785b9b65918aacc0a9a86b4d387966e81d: Status 404 returned error can't find the container with id 42dc52be9a9c33a66f774ecc61ecf8785b9b65918aacc0a9a86b4d387966e81d Jul 21 14:19:28.283962 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:28.283933 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:19:28.284030 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:28.284001 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:19:28.284065 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:28.284029 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:19:28.802909 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:28.802872 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" event={"ID":"922388ac-5be3-4163-bd9f-0e0fe10575e2","Type":"ContainerStarted","Data":"e2f8fc0438e54821ec8f22dafc163803562776d897ac3e647d81b8d60cbdb9ff"} Jul 21 14:19:28.803174 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:28.802933 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" event={"ID":"922388ac-5be3-4163-bd9f-0e0fe10575e2","Type":"ContainerStarted","Data":"42dc52be9a9c33a66f774ecc61ecf8785b9b65918aacc0a9a86b4d387966e81d"} Jul 21 14:19:28.830649 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:28.830591 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" podStartSLOduration=1.830572579 podStartE2EDuration="1.830572579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:19:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:19:28.828158139 +0000 UTC m=+2129.979999248" watchObservedRunningTime="2026-07-21 14:19:28.830572579 +0000 UTC m=+2129.982413687" Jul 21 14:19:29.121060 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:29.120951 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:29.135021 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:29.134980 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:29.811815 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:29.811779 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:29.817275 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:29.817237 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:30.745952 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:30.745916 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:30.849548 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:30.849478 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:30.935721 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:30.932237 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:31.141229 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:31.141129 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:31.430594 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:31.430555 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:31.533521 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:31.533477 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:35.231998 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:35.231947 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:36.762807 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:36.762768 2567 patch_prober.go:28] interesting pod/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 21 14:19:36.763253 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:36.762825 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" podUID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 21 14:19:36.860618 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:36.860578 2567 generic.go:356] "Generic (PLEG): container finished" podID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerID="74f784462dc9973e4310e119211e655d6ab145eb2911bd0adc269f46e8e1f5f9" exitCode=0 Jul 21 14:19:36.860834 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:36.860652 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" event={"ID":"cd80f796-9f59-4499-9012-9c77b6f9befd","Type":"ContainerDied","Data":"74f784462dc9973e4310e119211e655d6ab145eb2911bd0adc269f46e8e1f5f9"} Jul 21 14:19:36.861105 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:36.861091 2567 scope.go:117] "RemoveContainer" containerID="74f784462dc9973e4310e119211e655d6ab145eb2911bd0adc269f46e8e1f5f9" Jul 21 14:19:36.862509 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:36.862476 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:19:36.862576 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:36.862561 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:19:36.862613 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:36.862601 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:19:37.867886 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:37.867847 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" event={"ID":"cd80f796-9f59-4499-9012-9c77b6f9befd","Type":"ContainerStarted","Data":"d6ef5749adcd02e82e27f3c7c1c57439ea57aa587e468070e0414c48c81a13ca"} Jul 21 14:19:38.671892 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:38.671856 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:38.672115 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:38.672091 2567 patch_prober.go:28] interesting pod/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 21 14:19:38.672204 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:38.672149 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" podUID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 21 14:19:39.672426 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:39.672384 2567 patch_prober.go:28] interesting pod/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 21 14:19:39.672861 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:39.672447 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" podUID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 21 14:19:40.671751 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:40.671679 2567 patch_prober.go:28] interesting pod/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 21 14:19:40.673371 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:40.672068 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" podUID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 21 14:19:41.672137 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:41.672093 2567 patch_prober.go:28] interesting pod/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 21 14:19:41.672341 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:41.672157 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" podUID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 21 14:19:42.675882 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:42.675847 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:42.676398 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:42.676259 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:42.677068 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:42.677049 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:19:43.905723 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:43.905668 2567 generic.go:356] "Generic (PLEG): container finished" podID="922388ac-5be3-4163-bd9f-0e0fe10575e2" containerID="e2f8fc0438e54821ec8f22dafc163803562776d897ac3e647d81b8d60cbdb9ff" exitCode=0 Jul 21 14:19:43.906265 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:43.905744 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" event={"ID":"922388ac-5be3-4163-bd9f-0e0fe10575e2","Type":"ContainerDied","Data":"e2f8fc0438e54821ec8f22dafc163803562776d897ac3e647d81b8d60cbdb9ff"} Jul 21 14:19:43.906337 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:43.906296 2567 scope.go:117] "RemoveContainer" containerID="e2f8fc0438e54821ec8f22dafc163803562776d897ac3e647d81b8d60cbdb9ff" Jul 21 14:19:43.907318 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:43.907292 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:19:43.907421 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:43.907372 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:19:43.907421 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:43.907410 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:19:44.812423 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:44.812378 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:44.917723 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:44.917670 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" event={"ID":"922388ac-5be3-4163-bd9f-0e0fe10575e2","Type":"ContainerStarted","Data":"d42d87e26245a0a6cae4ea36c8de046b28662a964b7716ef271a747748c11224"} Jul 21 14:19:45.120893 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:45.120787 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:45.122224 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:45.122185 2567 patch_prober.go:28] interesting pod/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 21 14:19:45.122527 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:45.122496 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" podUID="922388ac-5be3-4163-bd9f-0e0fe10575e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 21 14:19:46.121432 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:46.121379 2567 patch_prober.go:28] interesting pod/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 21 14:19:46.121990 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:46.121474 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" podUID="922388ac-5be3-4163-bd9f-0e0fe10575e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 21 14:19:46.522102 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:46.522052 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:46.622281 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:46.622218 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:47.121466 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:47.121419 2567 patch_prober.go:28] interesting pod/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 21 14:19:47.122180 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:47.122137 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" podUID="922388ac-5be3-4163-bd9f-0e0fe10575e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 21 14:19:48.121368 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:48.121317 2567 patch_prober.go:28] interesting pod/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 21 14:19:48.121616 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:48.121391 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" podUID="922388ac-5be3-4163-bd9f-0e0fe10575e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 21 14:19:49.125536 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:49.125498 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:49.125975 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:49.125911 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:49.126776 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:49.126757 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:19:51.492749 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:51.492683 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:55.868912 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:55.868874 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:19:55.969250 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:19:55.969212 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:20:12.788195 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:20:12.788145 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:20:54.702685 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:20:54.702647 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:20:55.670134 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:20:55.670092 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:20:56.190312 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:20:56.190275 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:20:56.476224 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:20:56.476133 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:20:56.572111 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:20:56.572073 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:06.263214 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:06.263172 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:10.853302 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:10.853262 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:11.306754 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:11.306690 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:12.221274 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:12.221233 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:17.795818 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:17.795774 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:17.932806 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:17.932771 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:18.063572 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:18.063486 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh"] Jul 21 14:21:18.063739 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:18.063713 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" podUID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerName="istio-proxy" containerID="cri-o://d6ef5749adcd02e82e27f3c7c1c57439ea57aa587e468070e0414c48c81a13ca" gracePeriod=30 Jul 21 14:21:18.097243 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:18.097189 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:18.309631 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:18.309589 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:18.962919 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:18.962883 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:19.065189 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:19.065139 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:23.326902 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.326844 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:21:23.435508 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.435470 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-envoy\") pod \"cd80f796-9f59-4499-9012-9c77b6f9befd\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " Jul 21 14:21:23.435508 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.435515 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-workload-certs\") pod \"cd80f796-9f59-4499-9012-9c77b6f9befd\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " Jul 21 14:21:23.435776 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.435541 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-data\") pod \"cd80f796-9f59-4499-9012-9c77b6f9befd\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " Jul 21 14:21:23.435776 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.435576 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-workload-socket\") pod \"cd80f796-9f59-4499-9012-9c77b6f9befd\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " Jul 21 14:21:23.435776 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.435600 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd80f796-9f59-4499-9012-9c77b6f9befd-istiod-ca-cert\") pod \"cd80f796-9f59-4499-9012-9c77b6f9befd\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " Jul 21 14:21:23.435776 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.435624 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-token\") pod \"cd80f796-9f59-4499-9012-9c77b6f9befd\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " Jul 21 14:21:23.435776 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.435650 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-credential-socket\") pod \"cd80f796-9f59-4499-9012-9c77b6f9befd\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " Jul 21 14:21:23.435776 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.435678 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-podinfo\") pod \"cd80f796-9f59-4499-9012-9c77b6f9befd\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " Jul 21 14:21:23.436078 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.435830 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vscjb\" (UniqueName: \"kubernetes.io/projected/cd80f796-9f59-4499-9012-9c77b6f9befd-kube-api-access-vscjb\") pod \"cd80f796-9f59-4499-9012-9c77b6f9befd\" (UID: \"cd80f796-9f59-4499-9012-9c77b6f9befd\") " Jul 21 14:21:23.436078 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.435841 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cd80f796-9f59-4499-9012-9c77b6f9befd" (UID: "cd80f796-9f59-4499-9012-9c77b6f9befd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:21:23.436078 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.435954 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cd80f796-9f59-4499-9012-9c77b6f9befd" (UID: "cd80f796-9f59-4499-9012-9c77b6f9befd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:21:23.436078 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.436038 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cd80f796-9f59-4499-9012-9c77b6f9befd" (UID: "cd80f796-9f59-4499-9012-9c77b6f9befd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:21:23.436288 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.436132 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd80f796-9f59-4499-9012-9c77b6f9befd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cd80f796-9f59-4499-9012-9c77b6f9befd" (UID: "cd80f796-9f59-4499-9012-9c77b6f9befd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 14:21:23.436288 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.436205 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-workload-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:23.436288 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.436224 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-credential-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:23.436288 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.436237 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-workload-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:23.436625 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.436603 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-data" (OuterVolumeSpecName: "istio-data") pod "cd80f796-9f59-4499-9012-9c77b6f9befd" (UID: "cd80f796-9f59-4499-9012-9c77b6f9befd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:21:23.438154 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.438113 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd80f796-9f59-4499-9012-9c77b6f9befd-kube-api-access-vscjb" (OuterVolumeSpecName: "kube-api-access-vscjb") pod "cd80f796-9f59-4499-9012-9c77b6f9befd" (UID: "cd80f796-9f59-4499-9012-9c77b6f9befd"). InnerVolumeSpecName "kube-api-access-vscjb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:21:23.438154 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.438121 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-token" (OuterVolumeSpecName: "istio-token") pod "cd80f796-9f59-4499-9012-9c77b6f9befd" (UID: "cd80f796-9f59-4499-9012-9c77b6f9befd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:21:23.438353 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.438335 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cd80f796-9f59-4499-9012-9c77b6f9befd" (UID: "cd80f796-9f59-4499-9012-9c77b6f9befd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 14:21:23.438401 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.438347 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cd80f796-9f59-4499-9012-9c77b6f9befd" (UID: "cd80f796-9f59-4499-9012-9c77b6f9befd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:21:23.512215 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.512178 2567 generic.go:356] "Generic (PLEG): container finished" podID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerID="d6ef5749adcd02e82e27f3c7c1c57439ea57aa587e468070e0414c48c81a13ca" exitCode=0 Jul 21 14:21:23.512409 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.512225 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" Jul 21 14:21:23.512409 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.512263 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" event={"ID":"cd80f796-9f59-4499-9012-9c77b6f9befd","Type":"ContainerDied","Data":"d6ef5749adcd02e82e27f3c7c1c57439ea57aa587e468070e0414c48c81a13ca"} Jul 21 14:21:23.512409 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.512324 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh" event={"ID":"cd80f796-9f59-4499-9012-9c77b6f9befd","Type":"ContainerDied","Data":"cc601fcff8c9490ddb42c429e488964b6b5f71a065c25454207b2156e6452eda"} Jul 21 14:21:23.512409 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.512349 2567 scope.go:117] "RemoveContainer" containerID="d6ef5749adcd02e82e27f3c7c1c57439ea57aa587e468070e0414c48c81a13ca" Jul 21 14:21:23.523007 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.522987 2567 scope.go:117] "RemoveContainer" containerID="74f784462dc9973e4310e119211e655d6ab145eb2911bd0adc269f46e8e1f5f9" Jul 21 14:21:23.533097 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.533076 2567 scope.go:117] "RemoveContainer" containerID="d6ef5749adcd02e82e27f3c7c1c57439ea57aa587e468070e0414c48c81a13ca" Jul 21 14:21:23.533356 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:21:23.533336 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6ef5749adcd02e82e27f3c7c1c57439ea57aa587e468070e0414c48c81a13ca\": container with ID starting with d6ef5749adcd02e82e27f3c7c1c57439ea57aa587e468070e0414c48c81a13ca not found: ID does not exist" containerID="d6ef5749adcd02e82e27f3c7c1c57439ea57aa587e468070e0414c48c81a13ca" Jul 21 14:21:23.533429 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.533369 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6ef5749adcd02e82e27f3c7c1c57439ea57aa587e468070e0414c48c81a13ca"} err="failed to get container status \"d6ef5749adcd02e82e27f3c7c1c57439ea57aa587e468070e0414c48c81a13ca\": rpc error: code = NotFound desc = could not find container \"d6ef5749adcd02e82e27f3c7c1c57439ea57aa587e468070e0414c48c81a13ca\": container with ID starting with d6ef5749adcd02e82e27f3c7c1c57439ea57aa587e468070e0414c48c81a13ca not found: ID does not exist" Jul 21 14:21:23.533429 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.533396 2567 scope.go:117] "RemoveContainer" containerID="74f784462dc9973e4310e119211e655d6ab145eb2911bd0adc269f46e8e1f5f9" Jul 21 14:21:23.533629 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:21:23.533612 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74f784462dc9973e4310e119211e655d6ab145eb2911bd0adc269f46e8e1f5f9\": container with ID starting with 74f784462dc9973e4310e119211e655d6ab145eb2911bd0adc269f46e8e1f5f9 not found: ID does not exist" containerID="74f784462dc9973e4310e119211e655d6ab145eb2911bd0adc269f46e8e1f5f9" Jul 21 14:21:23.533666 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.533636 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74f784462dc9973e4310e119211e655d6ab145eb2911bd0adc269f46e8e1f5f9"} err="failed to get container status \"74f784462dc9973e4310e119211e655d6ab145eb2911bd0adc269f46e8e1f5f9\": rpc error: code = NotFound desc = could not find container \"74f784462dc9973e4310e119211e655d6ab145eb2911bd0adc269f46e8e1f5f9\": container with ID starting with 74f784462dc9973e4310e119211e655d6ab145eb2911bd0adc269f46e8e1f5f9 not found: ID does not exist" Jul 21 14:21:23.537409 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.537379 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-podinfo\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:23.537409 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.537408 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vscjb\" (UniqueName: \"kubernetes.io/projected/cd80f796-9f59-4499-9012-9c77b6f9befd-kube-api-access-vscjb\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:23.537568 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.537418 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-envoy\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:23.537568 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.537436 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-data\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:23.537568 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.537445 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd80f796-9f59-4499-9012-9c77b6f9befd-istiod-ca-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:23.537568 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.537457 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd80f796-9f59-4499-9012-9c77b6f9befd-istio-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:23.538583 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.538558 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh"] Jul 21 14:21:23.544367 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:23.544341 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-ce81f3-openshift-default-568fd95997-w7lhh"] Jul 21 14:21:25.533406 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:25.533367 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd80f796-9f59-4499-9012-9c77b6f9befd" path="/var/lib/kubelet/pods/cd80f796-9f59-4499-9012-9c77b6f9befd/volumes" Jul 21 14:21:28.961884 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:28.961841 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:33.503512 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:33.503468 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:33.868577 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:33.868464 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:40.271828 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:40.271778 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:40.525796 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:40.525676 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6"] Jul 21 14:21:40.526092 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:40.526055 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" podUID="922388ac-5be3-4163-bd9f-0e0fe10575e2" containerName="istio-proxy" containerID="cri-o://d42d87e26245a0a6cae4ea36c8de046b28662a964b7716ef271a747748c11224" gracePeriod=30 Jul 21 14:21:41.402771 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.402730 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx"] Jul 21 14:21:41.403360 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.403338 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerName="istio-proxy" Jul 21 14:21:41.403467 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.403363 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerName="istio-proxy" Jul 21 14:21:41.403467 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.403379 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerName="istio-proxy" Jul 21 14:21:41.403467 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.403396 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerName="istio-proxy" Jul 21 14:21:41.403631 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.403547 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerName="istio-proxy" Jul 21 14:21:41.403631 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.403562 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd80f796-9f59-4499-9012-9c77b6f9befd" containerName="istio-proxy" Jul 21 14:21:41.407096 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.407070 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.410506 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.410477 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-44f385-openshift-default-dockercfg-2f89z\"" Jul 21 14:21:41.421599 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.421558 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx"] Jul 21 14:21:41.520277 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.520242 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-token\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.520501 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.520312 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-podinfo\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.520501 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.520372 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istiod-ca-cert\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.520501 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.520459 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-envoy\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.520691 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.520498 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnb2x\" (UniqueName: \"kubernetes.io/projected/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-kube-api-access-qnb2x\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.520691 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.520539 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-credential-socket\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.520691 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.520610 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-workload-socket\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.520691 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.520634 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-data\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.520691 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.520663 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-workload-certs\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.622016 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.621956 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-workload-socket\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.622016 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.622013 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-data\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.622331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.622041 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-workload-certs\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.622331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.622121 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-token\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.622331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.622189 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-podinfo\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.622331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.622211 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istiod-ca-cert\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.622331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.622268 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-envoy\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.622331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.622294 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnb2x\" (UniqueName: \"kubernetes.io/projected/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-kube-api-access-qnb2x\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.622331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.622330 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-credential-socket\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.622738 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.622552 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-data\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.622738 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.622602 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-workload-socket\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.622861 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.622831 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-credential-socket\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.623026 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.623004 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-workload-certs\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.623521 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.623501 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istiod-ca-cert\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.625106 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.625086 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-podinfo\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.625245 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.625224 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-envoy\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.634158 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.634129 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-token\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.634657 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.634632 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnb2x\" (UniqueName: \"kubernetes.io/projected/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-kube-api-access-qnb2x\") pod \"e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.722223 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.722113 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:41.726833 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.726560 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:41.878578 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.878548 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx"] Jul 21 14:21:41.880550 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:21:41.880518 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod298ccaea_bacb_47a5_b92f_0c3e905f2ef5.slice/crio-01de74a9b5f0fd29311722dce5da6a61874d0f0d92d7f6534815a8dc21cd57de WatchSource:0}: Error finding container 01de74a9b5f0fd29311722dce5da6a61874d0f0d92d7f6534815a8dc21cd57de: Status 404 returned error can't find the container with id 01de74a9b5f0fd29311722dce5da6a61874d0f0d92d7f6534815a8dc21cd57de Jul 21 14:21:41.882796 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.882762 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:21:41.882893 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.882834 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:21:41.882893 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:41.882866 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:21:42.637913 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:42.637868 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" event={"ID":"298ccaea-bacb-47a5-b92f-0c3e905f2ef5","Type":"ContainerStarted","Data":"9b216afea7f88e258aaad61253a4ecdc189424d47e89f00f563d14334eb1ca38"} Jul 21 14:21:42.638410 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:42.637932 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" event={"ID":"298ccaea-bacb-47a5-b92f-0c3e905f2ef5","Type":"ContainerStarted","Data":"01de74a9b5f0fd29311722dce5da6a61874d0f0d92d7f6534815a8dc21cd57de"} Jul 21 14:21:42.660894 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:42.660832 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" podStartSLOduration=1.6608120290000001 podStartE2EDuration="1.660812029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:21:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:21:42.658176816 +0000 UTC m=+2263.810017924" watchObservedRunningTime="2026-07-21 14:21:42.660812029 +0000 UTC m=+2263.812653214" Jul 21 14:21:42.722675 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:42.722627 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:42.728222 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:42.728190 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:43.643796 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:43.643761 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:43.645065 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:43.645040 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:21:44.568829 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:44.568739 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:44.663607 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:44.663528 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:44.768757 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:44.768680 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:44.977980 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:44.977943 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:45.408422 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.408382 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:45.669794 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.669659 2567 generic.go:356] "Generic (PLEG): container finished" podID="922388ac-5be3-4163-bd9f-0e0fe10575e2" containerID="d42d87e26245a0a6cae4ea36c8de046b28662a964b7716ef271a747748c11224" exitCode=0 Jul 21 14:21:45.669794 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.669743 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" event={"ID":"922388ac-5be3-4163-bd9f-0e0fe10575e2","Type":"ContainerDied","Data":"d42d87e26245a0a6cae4ea36c8de046b28662a964b7716ef271a747748c11224"} Jul 21 14:21:45.670326 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.669853 2567 scope.go:117] "RemoveContainer" containerID="e2f8fc0438e54821ec8f22dafc163803562776d897ac3e647d81b8d60cbdb9ff" Jul 21 14:21:45.805641 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.805594 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:21:45.866428 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.866381 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-workload-certs\") pod \"922388ac-5be3-4163-bd9f-0e0fe10575e2\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " Jul 21 14:21:45.866627 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.866438 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-credential-socket\") pod \"922388ac-5be3-4163-bd9f-0e0fe10575e2\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " Jul 21 14:21:45.866627 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.866467 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-workload-socket\") pod \"922388ac-5be3-4163-bd9f-0e0fe10575e2\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " Jul 21 14:21:45.866627 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.866509 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v24k5\" (UniqueName: \"kubernetes.io/projected/922388ac-5be3-4163-bd9f-0e0fe10575e2-kube-api-access-v24k5\") pod \"922388ac-5be3-4163-bd9f-0e0fe10575e2\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " Jul 21 14:21:45.866627 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.866540 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-token\") pod \"922388ac-5be3-4163-bd9f-0e0fe10575e2\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " Jul 21 14:21:45.866627 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.866573 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922388ac-5be3-4163-bd9f-0e0fe10575e2-istiod-ca-cert\") pod \"922388ac-5be3-4163-bd9f-0e0fe10575e2\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " Jul 21 14:21:45.866627 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.866610 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-data\") pod \"922388ac-5be3-4163-bd9f-0e0fe10575e2\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " Jul 21 14:21:45.866959 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.866638 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-podinfo\") pod \"922388ac-5be3-4163-bd9f-0e0fe10575e2\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " Jul 21 14:21:45.866959 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.866775 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "922388ac-5be3-4163-bd9f-0e0fe10575e2" (UID: "922388ac-5be3-4163-bd9f-0e0fe10575e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:21:45.866959 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.866813 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-envoy\") pod \"922388ac-5be3-4163-bd9f-0e0fe10575e2\" (UID: \"922388ac-5be3-4163-bd9f-0e0fe10575e2\") " Jul 21 14:21:45.866959 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.866900 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "922388ac-5be3-4163-bd9f-0e0fe10575e2" (UID: "922388ac-5be3-4163-bd9f-0e0fe10575e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:21:45.867151 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.867089 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/922388ac-5be3-4163-bd9f-0e0fe10575e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "922388ac-5be3-4163-bd9f-0e0fe10575e2" (UID: "922388ac-5be3-4163-bd9f-0e0fe10575e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 14:21:45.867271 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.867249 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-workload-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:45.867354 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.867329 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "922388ac-5be3-4163-bd9f-0e0fe10575e2" (UID: "922388ac-5be3-4163-bd9f-0e0fe10575e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:21:45.867435 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.867401 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-credential-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:45.867435 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.867418 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/922388ac-5be3-4163-bd9f-0e0fe10575e2-istiod-ca-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:45.867605 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.867583 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "922388ac-5be3-4163-bd9f-0e0fe10575e2" (UID: "922388ac-5be3-4163-bd9f-0e0fe10575e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:21:45.869217 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.869187 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "922388ac-5be3-4163-bd9f-0e0fe10575e2" (UID: "922388ac-5be3-4163-bd9f-0e0fe10575e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:21:45.869357 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.869270 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "922388ac-5be3-4163-bd9f-0e0fe10575e2" (UID: "922388ac-5be3-4163-bd9f-0e0fe10575e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:21:45.869570 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.869543 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "922388ac-5be3-4163-bd9f-0e0fe10575e2" (UID: "922388ac-5be3-4163-bd9f-0e0fe10575e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 14:21:45.869570 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.869546 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/922388ac-5be3-4163-bd9f-0e0fe10575e2-kube-api-access-v24k5" (OuterVolumeSpecName: "kube-api-access-v24k5") pod "922388ac-5be3-4163-bd9f-0e0fe10575e2" (UID: "922388ac-5be3-4163-bd9f-0e0fe10575e2"). InnerVolumeSpecName "kube-api-access-v24k5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:21:45.968057 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.967946 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-envoy\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:45.968057 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.967983 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-workload-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:45.968057 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.967993 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v24k5\" (UniqueName: \"kubernetes.io/projected/922388ac-5be3-4163-bd9f-0e0fe10575e2-kube-api-access-v24k5\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:45.968057 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.968003 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:45.968057 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.968012 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-data\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:45.968057 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:45.968020 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/922388ac-5be3-4163-bd9f-0e0fe10575e2-istio-podinfo\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:21:46.678063 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:46.678018 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" event={"ID":"922388ac-5be3-4163-bd9f-0e0fe10575e2","Type":"ContainerDied","Data":"42dc52be9a9c33a66f774ecc61ecf8785b9b65918aacc0a9a86b4d387966e81d"} Jul 21 14:21:46.678063 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:46.678047 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6" Jul 21 14:21:46.678682 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:46.678095 2567 scope.go:117] "RemoveContainer" containerID="d42d87e26245a0a6cae4ea36c8de046b28662a964b7716ef271a747748c11224" Jul 21 14:21:46.705340 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:46.705281 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6"] Jul 21 14:21:46.710229 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:46.710179 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-faeda6-openshift-default-7d6599688f-tsrl6"] Jul 21 14:21:46.826636 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:46.826593 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:47.534940 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:47.534897 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="922388ac-5be3-4163-bd9f-0e0fe10575e2" path="/var/lib/kubelet/pods/922388ac-5be3-4163-bd9f-0e0fe10575e2/volumes" Jul 21 14:21:49.079853 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.079817 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:49.543163 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.543089 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:49.818039 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.817925 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm"] Jul 21 14:21:49.818523 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.818505 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="922388ac-5be3-4163-bd9f-0e0fe10575e2" containerName="istio-proxy" Jul 21 14:21:49.818610 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.818525 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="922388ac-5be3-4163-bd9f-0e0fe10575e2" containerName="istio-proxy" Jul 21 14:21:49.818647 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.818641 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="922388ac-5be3-4163-bd9f-0e0fe10575e2" containerName="istio-proxy" Jul 21 14:21:49.818682 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.818653 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="922388ac-5be3-4163-bd9f-0e0fe10575e2" containerName="istio-proxy" Jul 21 14:21:49.823819 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.823784 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:49.826849 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.826816 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ce2e87-openshift-default-dockercfg-2dq74\"" Jul 21 14:21:49.834843 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.834810 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm"] Jul 21 14:21:49.846125 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.846083 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:49.908830 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.907068 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-podinfo\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:49.908830 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.907174 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-workload-socket\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:49.908830 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.907310 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-workload-certs\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:49.908830 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.907345 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-token\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:49.908830 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.907420 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-credential-socket\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:49.908830 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.907466 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wljjc\" (UniqueName: \"kubernetes.io/projected/b52fed5b-0735-4ea5-9f44-e87ec0884644-kube-api-access-wljjc\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:49.908830 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.907515 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b52fed5b-0735-4ea5-9f44-e87ec0884644-istiod-ca-cert\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:49.908830 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.907550 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-data\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:49.908830 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:49.907635 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-envoy\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.009206 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.009163 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-workload-certs\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.009206 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.009209 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-token\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.009206 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.009235 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-credential-socket\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.009577 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.009255 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wljjc\" (UniqueName: \"kubernetes.io/projected/b52fed5b-0735-4ea5-9f44-e87ec0884644-kube-api-access-wljjc\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.009577 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.009284 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b52fed5b-0735-4ea5-9f44-e87ec0884644-istiod-ca-cert\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.009577 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.009315 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-data\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.009577 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.009380 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-envoy\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.009577 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.009405 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-podinfo\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.009577 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.009455 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-workload-socket\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.009950 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.009783 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-credential-socket\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.009950 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.009832 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-workload-socket\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.010066 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.010042 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-workload-certs\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.010132 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.010074 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-data\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.010367 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.010341 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b52fed5b-0735-4ea5-9f44-e87ec0884644-istiod-ca-cert\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.012188 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.012160 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-envoy\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.012465 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.012444 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-podinfo\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.018422 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.018374 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-token\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.018600 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.018508 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wljjc\" (UniqueName: \"kubernetes.io/projected/b52fed5b-0735-4ea5-9f44-e87ec0884644-kube-api-access-wljjc\") pod \"e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.041292 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.041252 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:50.138361 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.138246 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:50.302145 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.302113 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm"] Jul 21 14:21:50.303293 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:21:50.303266 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb52fed5b_0735_4ea5_9f44_e87ec0884644.slice/crio-d74978b2263171f126c9b435058935e01a1ebb90dc1f665c4a0a43c86a3afa45 WatchSource:0}: Error finding container d74978b2263171f126c9b435058935e01a1ebb90dc1f665c4a0a43c86a3afa45: Status 404 returned error can't find the container with id d74978b2263171f126c9b435058935e01a1ebb90dc1f665c4a0a43c86a3afa45 Jul 21 14:21:50.305569 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.305527 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:21:50.305666 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.305621 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:21:50.305737 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.305662 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:21:50.707830 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.707780 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" event={"ID":"b52fed5b-0735-4ea5-9f44-e87ec0884644","Type":"ContainerStarted","Data":"cb68fef88f831ae77b8f60d1011fac3d2d11f4d52ea1572ca6cc062effb612a4"} Jul 21 14:21:50.708087 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.707863 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" event={"ID":"b52fed5b-0735-4ea5-9f44-e87ec0884644","Type":"ContainerStarted","Data":"d74978b2263171f126c9b435058935e01a1ebb90dc1f665c4a0a43c86a3afa45"} Jul 21 14:21:50.737277 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:50.737207 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" podStartSLOduration=1.737183426 podStartE2EDuration="1.737183426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:21:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:21:50.734534871 +0000 UTC m=+2271.886375991" watchObservedRunningTime="2026-07-21 14:21:50.737183426 +0000 UTC m=+2271.889024537" Jul 21 14:21:51.138973 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:51.138854 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:51.140875 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:51.140837 2567 patch_prober.go:28] interesting pod/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 21 14:21:51.141058 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:51.140907 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 21 14:21:52.144235 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:52.144198 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:52.669640 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:52.669600 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:52.720534 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:52.720496 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:52.721808 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:52.721785 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:21:52.818366 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:52.818328 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:53.010605 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:53.010507 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:53.110749 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:53.110689 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:53.431336 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:53.431297 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:53.536972 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:53.536889 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:57.155994 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:57.155947 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:21:59.685930 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:59.685877 2567 patch_prober.go:28] interesting pod/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 14:21:59.686376 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:21:59.685990 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" podUID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 14:22:01.788686 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:01.788650 2567 generic.go:356] "Generic (PLEG): container finished" podID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerID="9b216afea7f88e258aaad61253a4ecdc189424d47e89f00f563d14334eb1ca38" exitCode=0 Jul 21 14:22:01.789142 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:01.788725 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" event={"ID":"298ccaea-bacb-47a5-b92f-0c3e905f2ef5","Type":"ContainerDied","Data":"9b216afea7f88e258aaad61253a4ecdc189424d47e89f00f563d14334eb1ca38"} Jul 21 14:22:01.789185 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:01.789161 2567 scope.go:117] "RemoveContainer" containerID="9b216afea7f88e258aaad61253a4ecdc189424d47e89f00f563d14334eb1ca38" Jul 21 14:22:01.790295 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:01.790265 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:22:01.790354 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:01.790339 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:22:01.790403 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:01.790390 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:22:02.796890 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:02.796838 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" event={"ID":"298ccaea-bacb-47a5-b92f-0c3e905f2ef5","Type":"ContainerStarted","Data":"0a28322e3b5efbf981eca5baff00a7c8a468c93858b4f8b3d497b8256a05e01b"} Jul 21 14:22:03.723235 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:03.723197 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:22:03.723644 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:03.723617 2567 patch_prober.go:28] interesting pod/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 21 14:22:03.723844 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:03.723813 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" podUID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 21 14:22:04.723164 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:04.723123 2567 patch_prober.go:28] interesting pod/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 21 14:22:04.724179 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:04.723764 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" podUID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 21 14:22:05.723136 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:05.723093 2567 patch_prober.go:28] interesting pod/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 21 14:22:05.723910 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:05.723455 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" podUID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 21 14:22:06.723661 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:06.723622 2567 patch_prober.go:28] interesting pod/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 21 14:22:06.724003 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:06.723963 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" podUID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 21 14:22:07.258370 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:07.258266 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:07.358174 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:07.358140 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:07.722567 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:07.722531 2567 patch_prober.go:28] interesting pod/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 21 14:22:07.722768 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:07.722593 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 21 14:22:07.726091 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:07.726062 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:22:07.726433 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:07.726337 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:22:07.726975 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:07.726961 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:22:07.835957 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:07.835928 2567 generic.go:356] "Generic (PLEG): container finished" podID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerID="cb68fef88f831ae77b8f60d1011fac3d2d11f4d52ea1572ca6cc062effb612a4" exitCode=0 Jul 21 14:22:07.836129 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:07.836001 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" event={"ID":"b52fed5b-0735-4ea5-9f44-e87ec0884644","Type":"ContainerDied","Data":"cb68fef88f831ae77b8f60d1011fac3d2d11f4d52ea1572ca6cc062effb612a4"} Jul 21 14:22:07.836531 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:07.836510 2567 scope.go:117] "RemoveContainer" containerID="cb68fef88f831ae77b8f60d1011fac3d2d11f4d52ea1572ca6cc062effb612a4" Jul 21 14:22:07.837460 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:07.837435 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:22:07.837519 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:07.837498 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:22:07.837556 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:07.837532 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:22:08.843909 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:08.843875 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" event={"ID":"b52fed5b-0735-4ea5-9f44-e87ec0884644","Type":"ContainerStarted","Data":"f2c02424dfbd5f60800d394c89d3bb804872bcd08ca146ca10e9ebfc70f89524"} Jul 21 14:22:09.138787 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:09.138661 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:22:09.138947 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:09.138919 2567 patch_prober.go:28] interesting pod/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 21 14:22:09.138999 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:09.138965 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 21 14:22:10.138649 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:10.138615 2567 patch_prober.go:28] interesting pod/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 21 14:22:10.139166 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:10.138673 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 21 14:22:11.138890 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:11.138846 2567 patch_prober.go:28] interesting pod/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 21 14:22:11.139335 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:11.138915 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 21 14:22:12.139349 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:12.139291 2567 patch_prober.go:28] interesting pod/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 21 14:22:12.139849 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:12.139370 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 21 14:22:13.142583 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:13.142547 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:22:13.143024 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:13.142921 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:22:13.143732 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:13.143713 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:22:15.281048 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:15.281015 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:15.381677 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:15.381636 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:20.555358 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:20.555315 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:25.265417 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:25.265372 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:25.661631 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:25.661598 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:31.744132 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:31.744075 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:31.823546 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:31.823489 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:31.991420 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:31.991362 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx"] Jul 21 14:22:31.991768 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:31.991736 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" podUID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerName="istio-proxy" containerID="cri-o://0a28322e3b5efbf981eca5baff00a7c8a468c93858b4f8b3d497b8256a05e01b" gracePeriod=30 Jul 21 14:22:32.015904 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:32.015763 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:32.276372 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:32.276264 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:37.294290 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.294223 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:22:37.405916 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.405865 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-token\") pod \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " Jul 21 14:22:37.406178 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.405963 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-podinfo\") pod \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " Jul 21 14:22:37.406178 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.405997 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-credential-socket\") pod \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " Jul 21 14:22:37.406178 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.406034 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-workload-certs\") pod \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " Jul 21 14:22:37.406178 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.406092 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-data\") pod \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " Jul 21 14:22:37.406178 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.406146 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istiod-ca-cert\") pod \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " Jul 21 14:22:37.406462 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.406198 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-envoy\") pod \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " Jul 21 14:22:37.406462 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.406256 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnb2x\" (UniqueName: \"kubernetes.io/projected/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-kube-api-access-qnb2x\") pod \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " Jul 21 14:22:37.406462 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.406319 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-workload-socket\") pod \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\" (UID: \"298ccaea-bacb-47a5-b92f-0c3e905f2ef5\") " Jul 21 14:22:37.406462 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.406347 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "298ccaea-bacb-47a5-b92f-0c3e905f2ef5" (UID: "298ccaea-bacb-47a5-b92f-0c3e905f2ef5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:22:37.406722 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.406680 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-credential-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:22:37.406997 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.406953 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "298ccaea-bacb-47a5-b92f-0c3e905f2ef5" (UID: "298ccaea-bacb-47a5-b92f-0c3e905f2ef5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:22:37.407168 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.407114 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "298ccaea-bacb-47a5-b92f-0c3e905f2ef5" (UID: "298ccaea-bacb-47a5-b92f-0c3e905f2ef5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:22:37.407333 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.407269 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "298ccaea-bacb-47a5-b92f-0c3e905f2ef5" (UID: "298ccaea-bacb-47a5-b92f-0c3e905f2ef5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 14:22:37.407472 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.407366 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-data" (OuterVolumeSpecName: "istio-data") pod "298ccaea-bacb-47a5-b92f-0c3e905f2ef5" (UID: "298ccaea-bacb-47a5-b92f-0c3e905f2ef5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:22:37.408822 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.408790 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "298ccaea-bacb-47a5-b92f-0c3e905f2ef5" (UID: "298ccaea-bacb-47a5-b92f-0c3e905f2ef5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 14:22:37.408961 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.408849 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-token" (OuterVolumeSpecName: "istio-token") pod "298ccaea-bacb-47a5-b92f-0c3e905f2ef5" (UID: "298ccaea-bacb-47a5-b92f-0c3e905f2ef5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:22:37.409068 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.409041 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "298ccaea-bacb-47a5-b92f-0c3e905f2ef5" (UID: "298ccaea-bacb-47a5-b92f-0c3e905f2ef5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:22:37.409556 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.409528 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-kube-api-access-qnb2x" (OuterVolumeSpecName: "kube-api-access-qnb2x") pod "298ccaea-bacb-47a5-b92f-0c3e905f2ef5" (UID: "298ccaea-bacb-47a5-b92f-0c3e905f2ef5"). InnerVolumeSpecName "kube-api-access-qnb2x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:22:37.507469 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.507421 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istiod-ca-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:22:37.507469 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.507462 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-envoy\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:22:37.507469 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.507476 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnb2x\" (UniqueName: \"kubernetes.io/projected/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-kube-api-access-qnb2x\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:22:37.507772 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.507491 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-workload-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:22:37.507772 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.507505 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:22:37.507772 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.507517 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-podinfo\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:22:37.507772 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.507529 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-workload-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:22:37.507772 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:37.507541 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/298ccaea-bacb-47a5-b92f-0c3e905f2ef5-istio-data\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:22:38.031592 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:38.031556 2567 generic.go:356] "Generic (PLEG): container finished" podID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerID="0a28322e3b5efbf981eca5baff00a7c8a468c93858b4f8b3d497b8256a05e01b" exitCode=0 Jul 21 14:22:38.031826 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:38.031614 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" Jul 21 14:22:38.031826 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:38.031649 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" event={"ID":"298ccaea-bacb-47a5-b92f-0c3e905f2ef5","Type":"ContainerDied","Data":"0a28322e3b5efbf981eca5baff00a7c8a468c93858b4f8b3d497b8256a05e01b"} Jul 21 14:22:38.031826 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:38.031751 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx" event={"ID":"298ccaea-bacb-47a5-b92f-0c3e905f2ef5","Type":"ContainerDied","Data":"01de74a9b5f0fd29311722dce5da6a61874d0f0d92d7f6534815a8dc21cd57de"} Jul 21 14:22:38.031826 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:38.031779 2567 scope.go:117] "RemoveContainer" containerID="0a28322e3b5efbf981eca5baff00a7c8a468c93858b4f8b3d497b8256a05e01b" Jul 21 14:22:38.042304 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:38.042279 2567 scope.go:117] "RemoveContainer" containerID="9b216afea7f88e258aaad61253a4ecdc189424d47e89f00f563d14334eb1ca38" Jul 21 14:22:38.054452 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:38.054205 2567 scope.go:117] "RemoveContainer" containerID="0a28322e3b5efbf981eca5baff00a7c8a468c93858b4f8b3d497b8256a05e01b" Jul 21 14:22:38.054683 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:22:38.054659 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a28322e3b5efbf981eca5baff00a7c8a468c93858b4f8b3d497b8256a05e01b\": container with ID starting with 0a28322e3b5efbf981eca5baff00a7c8a468c93858b4f8b3d497b8256a05e01b not found: ID does not exist" containerID="0a28322e3b5efbf981eca5baff00a7c8a468c93858b4f8b3d497b8256a05e01b" Jul 21 14:22:38.054784 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:38.054712 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a28322e3b5efbf981eca5baff00a7c8a468c93858b4f8b3d497b8256a05e01b"} err="failed to get container status \"0a28322e3b5efbf981eca5baff00a7c8a468c93858b4f8b3d497b8256a05e01b\": rpc error: code = NotFound desc = could not find container \"0a28322e3b5efbf981eca5baff00a7c8a468c93858b4f8b3d497b8256a05e01b\": container with ID starting with 0a28322e3b5efbf981eca5baff00a7c8a468c93858b4f8b3d497b8256a05e01b not found: ID does not exist" Jul 21 14:22:38.054784 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:38.054735 2567 scope.go:117] "RemoveContainer" containerID="9b216afea7f88e258aaad61253a4ecdc189424d47e89f00f563d14334eb1ca38" Jul 21 14:22:38.054977 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:38.054959 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx"] Jul 21 14:22:38.055037 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:22:38.054992 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b216afea7f88e258aaad61253a4ecdc189424d47e89f00f563d14334eb1ca38\": container with ID starting with 9b216afea7f88e258aaad61253a4ecdc189424d47e89f00f563d14334eb1ca38 not found: ID does not exist" containerID="9b216afea7f88e258aaad61253a4ecdc189424d47e89f00f563d14334eb1ca38" Jul 21 14:22:38.055037 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:38.055020 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b216afea7f88e258aaad61253a4ecdc189424d47e89f00f563d14334eb1ca38"} err="failed to get container status \"9b216afea7f88e258aaad61253a4ecdc189424d47e89f00f563d14334eb1ca38\": rpc error: code = NotFound desc = could not find container \"9b216afea7f88e258aaad61253a4ecdc189424d47e89f00f563d14334eb1ca38\": container with ID starting with 9b216afea7f88e258aaad61253a4ecdc189424d47e89f00f563d14334eb1ca38 not found: ID does not exist" Jul 21 14:22:38.057678 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:38.057646 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-44f385-openshift-default-dfdcb884c-m7ztx"] Jul 21 14:22:39.534443 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:39.534394 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" path="/var/lib/kubelet/pods/298ccaea-bacb-47a5-b92f-0c3e905f2ef5/volumes" Jul 21 14:22:47.985866 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:47.985824 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:52.469407 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:52.469315 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:52.858849 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:52.858749 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:55.321160 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:55.321094 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:58.913096 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:58.913054 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:58.980937 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:58.980872 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:59.157622 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:59.157570 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm"] Jul 21 14:22:59.157937 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:59.157884 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerName="istio-proxy" containerID="cri-o://f2c02424dfbd5f60800d394c89d3bb804872bcd08ca146ca10e9ebfc70f89524" gracePeriod=30 Jul 21 14:22:59.180357 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:59.180265 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:59.417200 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:59.417157 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:22:59.775342 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:22:59.775293 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:00.040644 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.040539 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx"] Jul 21 14:23:00.041049 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.040989 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="922388ac-5be3-4163-bd9f-0e0fe10575e2" containerName="istio-proxy" Jul 21 14:23:00.041049 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.041005 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="922388ac-5be3-4163-bd9f-0e0fe10575e2" containerName="istio-proxy" Jul 21 14:23:00.041049 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.041022 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerName="istio-proxy" Jul 21 14:23:00.041049 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.041028 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerName="istio-proxy" Jul 21 14:23:00.041049 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.041036 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerName="istio-proxy" Jul 21 14:23:00.041049 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.041042 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerName="istio-proxy" Jul 21 14:23:00.041242 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.041118 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerName="istio-proxy" Jul 21 14:23:00.041242 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.041128 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="298ccaea-bacb-47a5-b92f-0c3e905f2ef5" containerName="istio-proxy" Jul 21 14:23:00.045930 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.045898 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.048889 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.048861 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-67be07-openshift-default-dockercfg-gfxxg\"" Jul 21 14:23:00.062750 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.062714 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx"] Jul 21 14:23:00.072742 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.072682 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:00.117954 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.117909 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-workload-socket\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.118193 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.117966 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-envoy\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.118193 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.118040 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istiod-ca-cert\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.118193 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.118160 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-podinfo\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.118379 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.118198 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-data\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.118379 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.118232 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8hjb\" (UniqueName: \"kubernetes.io/projected/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-kube-api-access-g8hjb\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.118379 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.118307 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-credential-socket\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.118379 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.118345 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-token\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.118559 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.118405 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-workload-certs\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.141844 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.141788 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:00.219348 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.219299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-podinfo\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.219601 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.219366 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-data\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.219601 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.219482 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8hjb\" (UniqueName: \"kubernetes.io/projected/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-kube-api-access-g8hjb\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.219746 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.219534 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-credential-socket\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.219746 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.219645 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-token\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.219865 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.219843 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-workload-certs\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.219920 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.219894 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-data\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.220009 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.219968 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-workload-socket\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.220159 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.220034 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-envoy\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.220159 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.220065 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istiod-ca-cert\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.220301 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.220238 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-credential-socket\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.220365 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.220319 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-workload-socket\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.220554 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.220532 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-workload-certs\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.220952 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.220913 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istiod-ca-cert\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.223042 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.223016 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-envoy\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.223158 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.223140 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-podinfo\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.229574 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.229532 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-token\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.229783 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.229641 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8hjb\" (UniqueName: \"kubernetes.io/projected/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-kube-api-access-g8hjb\") pod \"e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.360786 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.360639 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:00.524015 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.523982 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx"] Jul 21 14:23:00.526017 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:23:00.525980 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e75e1d0_15a7_4372_8fe8_40dc19bcc06b.slice/crio-7515f31a7026bd8cd7f6b4f45314381832d6d562d253592097e33aea6a5160ca WatchSource:0}: Error finding container 7515f31a7026bd8cd7f6b4f45314381832d6d562d253592097e33aea6a5160ca: Status 404 returned error can't find the container with id 7515f31a7026bd8cd7f6b4f45314381832d6d562d253592097e33aea6a5160ca Jul 21 14:23:00.528325 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.528290 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:23:00.528451 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.528376 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:23:00.528451 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:00.528424 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:23:01.182457 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:01.182416 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" event={"ID":"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b","Type":"ContainerStarted","Data":"b1c857159a390e20f018b0175391f1ad489f5201885b78b945c23f9dfd51d33c"} Jul 21 14:23:01.183031 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:01.182492 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" event={"ID":"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b","Type":"ContainerStarted","Data":"7515f31a7026bd8cd7f6b4f45314381832d6d562d253592097e33aea6a5160ca"} Jul 21 14:23:01.206652 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:01.206590 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" podStartSLOduration=1.206571996 podStartE2EDuration="1.206571996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:23:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:23:01.203508599 +0000 UTC m=+2342.355349709" watchObservedRunningTime="2026-07-21 14:23:01.206571996 +0000 UTC m=+2342.358413104" Jul 21 14:23:01.361381 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:01.361339 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:01.366792 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:01.366757 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:02.189106 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:02.189056 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:02.190426 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:02.190397 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:03.125615 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:03.125575 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:03.230623 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:03.230579 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:03.410909 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:03.410861 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:03.650479 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:03.650419 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:03.945762 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:03.945719 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:04.044931 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.044895 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:04.455921 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.455871 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:23:04.566474 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.566343 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-data\") pod \"b52fed5b-0735-4ea5-9f44-e87ec0884644\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " Jul 21 14:23:04.566474 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.566414 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wljjc\" (UniqueName: \"kubernetes.io/projected/b52fed5b-0735-4ea5-9f44-e87ec0884644-kube-api-access-wljjc\") pod \"b52fed5b-0735-4ea5-9f44-e87ec0884644\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " Jul 21 14:23:04.566474 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.566466 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-workload-certs\") pod \"b52fed5b-0735-4ea5-9f44-e87ec0884644\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " Jul 21 14:23:04.566823 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.566490 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-token\") pod \"b52fed5b-0735-4ea5-9f44-e87ec0884644\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " Jul 21 14:23:04.566823 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.566584 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b52fed5b-0735-4ea5-9f44-e87ec0884644-istiod-ca-cert\") pod \"b52fed5b-0735-4ea5-9f44-e87ec0884644\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " Jul 21 14:23:04.566823 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.566732 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-workload-socket\") pod \"b52fed5b-0735-4ea5-9f44-e87ec0884644\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " Jul 21 14:23:04.566823 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.566774 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-credential-socket\") pod \"b52fed5b-0735-4ea5-9f44-e87ec0884644\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " Jul 21 14:23:04.566823 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.566801 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-envoy\") pod \"b52fed5b-0735-4ea5-9f44-e87ec0884644\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " Jul 21 14:23:04.567069 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.566882 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-podinfo\") pod \"b52fed5b-0735-4ea5-9f44-e87ec0884644\" (UID: \"b52fed5b-0735-4ea5-9f44-e87ec0884644\") " Jul 21 14:23:04.567287 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.566881 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b52fed5b-0735-4ea5-9f44-e87ec0884644" (UID: "b52fed5b-0735-4ea5-9f44-e87ec0884644"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:23:04.567287 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.567174 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b52fed5b-0735-4ea5-9f44-e87ec0884644" (UID: "b52fed5b-0735-4ea5-9f44-e87ec0884644"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:23:04.567511 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.567210 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b52fed5b-0735-4ea5-9f44-e87ec0884644-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b52fed5b-0735-4ea5-9f44-e87ec0884644" (UID: "b52fed5b-0735-4ea5-9f44-e87ec0884644"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 14:23:04.567511 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.567229 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b52fed5b-0735-4ea5-9f44-e87ec0884644" (UID: "b52fed5b-0735-4ea5-9f44-e87ec0884644"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:23:04.567511 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.567419 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-data" (OuterVolumeSpecName: "istio-data") pod "b52fed5b-0735-4ea5-9f44-e87ec0884644" (UID: "b52fed5b-0735-4ea5-9f44-e87ec0884644"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:23:04.569453 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.569405 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b52fed5b-0735-4ea5-9f44-e87ec0884644" (UID: "b52fed5b-0735-4ea5-9f44-e87ec0884644"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 14:23:04.569680 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.569404 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b52fed5b-0735-4ea5-9f44-e87ec0884644" (UID: "b52fed5b-0735-4ea5-9f44-e87ec0884644"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:23:04.569680 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.569548 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b52fed5b-0735-4ea5-9f44-e87ec0884644-kube-api-access-wljjc" (OuterVolumeSpecName: "kube-api-access-wljjc") pod "b52fed5b-0735-4ea5-9f44-e87ec0884644" (UID: "b52fed5b-0735-4ea5-9f44-e87ec0884644"). InnerVolumeSpecName "kube-api-access-wljjc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:23:04.569680 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.569542 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-token" (OuterVolumeSpecName: "istio-token") pod "b52fed5b-0735-4ea5-9f44-e87ec0884644" (UID: "b52fed5b-0735-4ea5-9f44-e87ec0884644"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:23:04.667864 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.667818 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b52fed5b-0735-4ea5-9f44-e87ec0884644-istiod-ca-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:23:04.667864 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.667855 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-workload-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:23:04.667864 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.667864 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-credential-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:23:04.667864 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.667873 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-envoy\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:23:04.667864 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.667882 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-podinfo\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:23:04.668204 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.667890 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-data\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:23:04.668204 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.667898 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wljjc\" (UniqueName: \"kubernetes.io/projected/b52fed5b-0735-4ea5-9f44-e87ec0884644-kube-api-access-wljjc\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:23:04.668204 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.667909 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b52fed5b-0735-4ea5-9f44-e87ec0884644-workload-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:23:04.668204 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:04.667917 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b52fed5b-0735-4ea5-9f44-e87ec0884644-istio-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:23:05.210925 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:05.210886 2567 generic.go:356] "Generic (PLEG): container finished" podID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerID="f2c02424dfbd5f60800d394c89d3bb804872bcd08ca146ca10e9ebfc70f89524" exitCode=0 Jul 21 14:23:05.211130 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:05.210957 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" Jul 21 14:23:05.211130 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:05.210976 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" event={"ID":"b52fed5b-0735-4ea5-9f44-e87ec0884644","Type":"ContainerDied","Data":"f2c02424dfbd5f60800d394c89d3bb804872bcd08ca146ca10e9ebfc70f89524"} Jul 21 14:23:05.211130 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:05.211048 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm" event={"ID":"b52fed5b-0735-4ea5-9f44-e87ec0884644","Type":"ContainerDied","Data":"d74978b2263171f126c9b435058935e01a1ebb90dc1f665c4a0a43c86a3afa45"} Jul 21 14:23:05.211130 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:05.211081 2567 scope.go:117] "RemoveContainer" containerID="f2c02424dfbd5f60800d394c89d3bb804872bcd08ca146ca10e9ebfc70f89524" Jul 21 14:23:05.222317 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:05.222293 2567 scope.go:117] "RemoveContainer" containerID="cb68fef88f831ae77b8f60d1011fac3d2d11f4d52ea1572ca6cc062effb612a4" Jul 21 14:23:05.234568 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:05.234542 2567 scope.go:117] "RemoveContainer" containerID="f2c02424dfbd5f60800d394c89d3bb804872bcd08ca146ca10e9ebfc70f89524" Jul 21 14:23:05.234959 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:23:05.234935 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2c02424dfbd5f60800d394c89d3bb804872bcd08ca146ca10e9ebfc70f89524\": container with ID starting with f2c02424dfbd5f60800d394c89d3bb804872bcd08ca146ca10e9ebfc70f89524 not found: ID does not exist" containerID="f2c02424dfbd5f60800d394c89d3bb804872bcd08ca146ca10e9ebfc70f89524" Jul 21 14:23:05.235058 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:05.234967 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2c02424dfbd5f60800d394c89d3bb804872bcd08ca146ca10e9ebfc70f89524"} err="failed to get container status \"f2c02424dfbd5f60800d394c89d3bb804872bcd08ca146ca10e9ebfc70f89524\": rpc error: code = NotFound desc = could not find container \"f2c02424dfbd5f60800d394c89d3bb804872bcd08ca146ca10e9ebfc70f89524\": container with ID starting with f2c02424dfbd5f60800d394c89d3bb804872bcd08ca146ca10e9ebfc70f89524 not found: ID does not exist" Jul 21 14:23:05.235058 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:05.234984 2567 scope.go:117] "RemoveContainer" containerID="cb68fef88f831ae77b8f60d1011fac3d2d11f4d52ea1572ca6cc062effb612a4" Jul 21 14:23:05.235268 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:23:05.235250 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb68fef88f831ae77b8f60d1011fac3d2d11f4d52ea1572ca6cc062effb612a4\": container with ID starting with cb68fef88f831ae77b8f60d1011fac3d2d11f4d52ea1572ca6cc062effb612a4 not found: ID does not exist" containerID="cb68fef88f831ae77b8f60d1011fac3d2d11f4d52ea1572ca6cc062effb612a4" Jul 21 14:23:05.235314 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:05.235275 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb68fef88f831ae77b8f60d1011fac3d2d11f4d52ea1572ca6cc062effb612a4"} err="failed to get container status \"cb68fef88f831ae77b8f60d1011fac3d2d11f4d52ea1572ca6cc062effb612a4\": rpc error: code = NotFound desc = could not find container \"cb68fef88f831ae77b8f60d1011fac3d2d11f4d52ea1572ca6cc062effb612a4\": container with ID starting with cb68fef88f831ae77b8f60d1011fac3d2d11f4d52ea1572ca6cc062effb612a4 not found: ID does not exist" Jul 21 14:23:05.240014 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:05.239967 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm"] Jul 21 14:23:05.246229 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:05.246189 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ce2e87-openshift-default-6864b4b9bf-km2sm"] Jul 21 14:23:05.535905 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:05.535788 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" path="/var/lib/kubelet/pods/b52fed5b-0735-4ea5-9f44-e87ec0884644/volumes" Jul 21 14:23:07.610181 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:07.610134 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:07.936788 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:07.936751 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:08.231479 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.231374 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn"] Jul 21 14:23:08.232056 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.232030 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerName="istio-proxy" Jul 21 14:23:08.232056 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.232058 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerName="istio-proxy" Jul 21 14:23:08.232231 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.232098 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerName="istio-proxy" Jul 21 14:23:08.232231 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.232108 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerName="istio-proxy" Jul 21 14:23:08.232304 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.232236 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerName="istio-proxy" Jul 21 14:23:08.232304 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.232250 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b52fed5b-0735-4ea5-9f44-e87ec0884644" containerName="istio-proxy" Jul 21 14:23:08.236548 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.236520 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.239732 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.239667 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-52967c-openshift-default-dockercfg-xvplj\"" Jul 21 14:23:08.251453 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.251396 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn"] Jul 21 14:23:08.267499 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.267465 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:08.305100 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.305030 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-workload-certs\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.305100 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.305083 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-envoy\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.305449 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.305156 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2hwr\" (UniqueName: \"kubernetes.io/projected/08375ad4-b016-4f95-af61-bdaa2b07c490-kube-api-access-p2hwr\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.305449 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.305199 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-credential-socket\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.305449 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.305320 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-token\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.305449 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.305374 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-podinfo\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.305449 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.305437 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08375ad4-b016-4f95-af61-bdaa2b07c490-istiod-ca-cert\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.305759 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.305468 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-workload-socket\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.305759 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.305502 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-data\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.365546 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.365501 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:08.406837 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.406792 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-token\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.407077 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.406852 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-podinfo\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.407077 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.406900 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08375ad4-b016-4f95-af61-bdaa2b07c490-istiod-ca-cert\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.407077 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.406930 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-workload-socket\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.407077 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.406962 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-data\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.407077 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.407018 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-workload-certs\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.407077 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.407041 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-envoy\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.407077 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.407070 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2hwr\" (UniqueName: \"kubernetes.io/projected/08375ad4-b016-4f95-af61-bdaa2b07c490-kube-api-access-p2hwr\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.407481 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.407130 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-credential-socket\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.407887 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.407810 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-data\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.407887 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.407830 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08375ad4-b016-4f95-af61-bdaa2b07c490-istiod-ca-cert\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.408676 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.408620 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-workload-socket\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.409007 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.408961 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-workload-certs\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.414816 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.410729 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-podinfo\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.414816 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.411694 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-envoy\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.414816 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.409335 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-credential-socket\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.417309 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.417277 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2hwr\" (UniqueName: \"kubernetes.io/projected/08375ad4-b016-4f95-af61-bdaa2b07c490-kube-api-access-p2hwr\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.418085 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.418057 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-token\") pod \"e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.549843 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.549734 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:08.717092 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.717062 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn"] Jul 21 14:23:08.719459 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:23:08.719428 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08375ad4_b016_4f95_af61_bdaa2b07c490.slice/crio-470696633ae56fa3e5bc492392ae656cc70090251aa65e0fcd310e43a1e0f4ce WatchSource:0}: Error finding container 470696633ae56fa3e5bc492392ae656cc70090251aa65e0fcd310e43a1e0f4ce: Status 404 returned error can't find the container with id 470696633ae56fa3e5bc492392ae656cc70090251aa65e0fcd310e43a1e0f4ce Jul 21 14:23:08.722387 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.722348 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:23:08.722499 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.722448 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:23:08.722499 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:08.722482 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:23:09.242805 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:09.242755 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" event={"ID":"08375ad4-b016-4f95-af61-bdaa2b07c490","Type":"ContainerStarted","Data":"7eeee1b4a5422ad6a2fb2dbac7e86a3ae0efe344c9f247714fbb3e4bbd921e47"} Jul 21 14:23:09.243030 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:09.242833 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" event={"ID":"08375ad4-b016-4f95-af61-bdaa2b07c490","Type":"ContainerStarted","Data":"470696633ae56fa3e5bc492392ae656cc70090251aa65e0fcd310e43a1e0f4ce"} Jul 21 14:23:09.268297 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:09.268231 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" podStartSLOduration=1.2682078620000001 podStartE2EDuration="1.268207862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:23:09.266841895 +0000 UTC m=+2350.418683008" watchObservedRunningTime="2026-07-21 14:23:09.268207862 +0000 UTC m=+2350.420048970" Jul 21 14:23:09.550859 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:09.550738 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:09.552458 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:09.552412 2567 patch_prober.go:28] interesting pod/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 21 14:23:09.552676 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:09.552491 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 21 14:23:10.554633 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:10.554549 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:11.241371 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:11.241319 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:11.255469 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:11.255434 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:11.256741 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:11.256685 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:11.480687 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:11.480651 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:11.581994 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:11.581883 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:12.032515 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:12.032473 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:12.139142 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:12.139100 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:15.647683 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:15.647647 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:18.230968 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:18.230926 2567 patch_prober.go:28] interesting pod/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 14:23:18.231505 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:18.230998 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 14:23:20.316780 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:20.316747 2567 generic.go:356] "Generic (PLEG): container finished" podID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerID="b1c857159a390e20f018b0175391f1ad489f5201885b78b945c23f9dfd51d33c" exitCode=0 Jul 21 14:23:20.317286 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:20.316826 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" event={"ID":"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b","Type":"ContainerDied","Data":"b1c857159a390e20f018b0175391f1ad489f5201885b78b945c23f9dfd51d33c"} Jul 21 14:23:20.317353 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:20.317307 2567 scope.go:117] "RemoveContainer" containerID="b1c857159a390e20f018b0175391f1ad489f5201885b78b945c23f9dfd51d33c" Jul 21 14:23:20.318454 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:20.318419 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:23:20.318562 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:20.318518 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:23:20.318562 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:20.318553 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:23:21.332079 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:21.332039 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" event={"ID":"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b","Type":"ContainerStarted","Data":"0467424b7093b5693067157f616ce9a5207d7ee121d1ab14f62ce8ed2630d01b"} Jul 21 14:23:21.360780 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:21.360745 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:21.361031 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:21.360989 2567 patch_prober.go:28] interesting pod/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 21 14:23:21.361183 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:21.361053 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 21 14:23:22.361478 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:22.361432 2567 patch_prober.go:28] interesting pod/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 21 14:23:22.361911 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:22.361504 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 21 14:23:23.361914 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:23.361862 2567 patch_prober.go:28] interesting pod/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 21 14:23:23.362356 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:23.361929 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 21 14:23:23.510725 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:23.510670 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:24.361066 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:24.361029 2567 patch_prober.go:28] interesting pod/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 21 14:23:24.361372 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:24.361340 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 21 14:23:24.716095 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:24.716033 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:24.813953 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:24.813739 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:25.361546 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:25.361485 2567 patch_prober.go:28] interesting pod/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 21 14:23:25.361749 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:25.361579 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 21 14:23:26.365452 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:26.365419 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:26.365880 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:26.365708 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:26.366494 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:26.366476 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:23:26.743384 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:26.743343 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:26.875776 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:26.875736 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:27.297929 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:27.297879 2567 patch_prober.go:28] interesting pod/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 14:23:27.298121 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:27.297952 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 14:23:29.395091 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:29.395056 2567 generic.go:356] "Generic (PLEG): container finished" podID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerID="7eeee1b4a5422ad6a2fb2dbac7e86a3ae0efe344c9f247714fbb3e4bbd921e47" exitCode=0 Jul 21 14:23:29.395639 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:29.395136 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" event={"ID":"08375ad4-b016-4f95-af61-bdaa2b07c490","Type":"ContainerDied","Data":"7eeee1b4a5422ad6a2fb2dbac7e86a3ae0efe344c9f247714fbb3e4bbd921e47"} Jul 21 14:23:29.395734 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:29.395676 2567 scope.go:117] "RemoveContainer" containerID="7eeee1b4a5422ad6a2fb2dbac7e86a3ae0efe344c9f247714fbb3e4bbd921e47" Jul 21 14:23:29.396804 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:29.396778 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:23:29.396897 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:29.396857 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:23:29.396936 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:29.396898 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 14:23:29.465489 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:29.465429 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:29.564749 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:29.564711 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:30.402740 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:30.402681 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" event={"ID":"08375ad4-b016-4f95-af61-bdaa2b07c490","Type":"ContainerStarted","Data":"d2b13be3f98f993acd08dc86afa64a4e692bfd3fbdc6a0d0dc51251a656af2e1"} Jul 21 14:23:30.550874 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:30.550833 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:30.551140 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:30.551119 2567 patch_prober.go:28] interesting pod/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 21 14:23:30.551198 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:30.551173 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 21 14:23:31.550349 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:31.550303 2567 patch_prober.go:28] interesting pod/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 21 14:23:31.550838 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:31.550375 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 21 14:23:32.438723 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:32.438620 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:32.550526 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:32.550480 2567 patch_prober.go:28] interesting pod/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 21 14:23:32.550964 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:32.550564 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 21 14:23:32.962197 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:32.962142 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:33.062313 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:33.062275 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:23:33.550180 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:33.550137 2567 patch_prober.go:28] interesting pod/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 21 14:23:33.550377 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:33.550204 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 21 14:23:34.550815 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:34.550772 2567 patch_prober.go:28] interesting pod/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 21 14:23:34.551369 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:34.550861 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 21 14:23:35.553810 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:35.553770 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:35.554267 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:35.554092 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:35.554851 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:35.554833 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:23:59.580710 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:59.580667 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:23:59.589987 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:23:59.589961 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:24:03.684091 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.684049 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr"] Jul 21 14:24:03.689314 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.689285 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.692262 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.692227 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-52967c\"/\"kube-root-ca.crt\"" Jul 21 14:24:03.694126 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.694019 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-52967c\"/\"openshift-service-ca.crt\"" Jul 21 14:24:03.694126 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.694064 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-52967c\"/\"default-dockercfg-qfcs7\"" Jul 21 14:24:03.694126 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.694078 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-52967c\"/\"auth-test-model-52967c-kserve-self-signed-certs\"" Jul 21 14:24:03.697991 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.697962 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr"] Jul 21 14:24:03.738131 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.738083 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:24:03.867013 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.866950 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-home\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.867250 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.867052 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75e95704-2138-4b63-a563-d08fbc924691-tls-certs\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.867250 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.867120 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-tmp-dir\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.867250 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.867155 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-dshm\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.867250 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.867194 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwfnx\" (UniqueName: \"kubernetes.io/projected/75e95704-2138-4b63-a563-d08fbc924691-kube-api-access-dwfnx\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.867443 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.867265 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-model-cache\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.968328 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.968199 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-home\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.968328 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.968272 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75e95704-2138-4b63-a563-d08fbc924691-tls-certs\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.968328 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.968299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-tmp-dir\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.968328 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.968317 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-dshm\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.968689 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.968344 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwfnx\" (UniqueName: \"kubernetes.io/projected/75e95704-2138-4b63-a563-d08fbc924691-kube-api-access-dwfnx\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.968689 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.968391 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-model-cache\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.968861 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.968776 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-home\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.968861 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.968815 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-tmp-dir\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.968861 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.968852 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-model-cache\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.970766 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.970732 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-dshm\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.970940 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.970924 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75e95704-2138-4b63-a563-d08fbc924691-tls-certs\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:03.978673 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:03.978647 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwfnx\" (UniqueName: \"kubernetes.io/projected/75e95704-2138-4b63-a563-d08fbc924691-kube-api-access-dwfnx\") pod \"auth-test-model-52967c-kserve-d57bd85c9-2fbfr\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:04.001612 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:04.001567 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:04.156793 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:04.156764 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr"] Jul 21 14:24:04.160529 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:24:04.160500 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75e95704_2138_4b63_a563_d08fbc924691.slice/crio-7656245b33be469a2c6192da4d85deac762023b00b7193c0c7829b2aacb868f2 WatchSource:0}: Error finding container 7656245b33be469a2c6192da4d85deac762023b00b7193c0c7829b2aacb868f2: Status 404 returned error can't find the container with id 7656245b33be469a2c6192da4d85deac762023b00b7193c0c7829b2aacb868f2 Jul 21 14:24:04.621314 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:04.621214 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" event={"ID":"75e95704-2138-4b63-a563-d08fbc924691","Type":"ContainerStarted","Data":"3399b2af94c74780dbd88a8be2e2a690912fd541233cfe6898992924354bf7c2"} Jul 21 14:24:04.621314 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:04.621266 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" event={"ID":"75e95704-2138-4b63-a563-d08fbc924691","Type":"ContainerStarted","Data":"7656245b33be469a2c6192da4d85deac762023b00b7193c0c7829b2aacb868f2"} Jul 21 14:24:04.639995 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:04.639932 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" podStartSLOduration=1.639908892 podStartE2EDuration="1.639908892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:24:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:24:04.637869583 +0000 UTC m=+2405.789710692" watchObservedRunningTime="2026-07-21 14:24:04.639908892 +0000 UTC m=+2405.791750003" Jul 21 14:24:14.001782 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:14.001723 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:14.002261 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:14.001802 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:14.015374 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:14.015336 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:24:14.693344 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:24:14.693313 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:26:10.558762 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:10.558638 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:26:11.700755 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:11.700712 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr"] Jul 21 14:26:11.710641 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:11.710597 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:26:22.228797 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:22.228755 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:26:26.724071 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:26.724008 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" podUID="75e95704-2138-4b63-a563-d08fbc924691" containerName="main" containerID="cri-o://3399b2af94c74780dbd88a8be2e2a690912fd541233cfe6898992924354bf7c2" gracePeriod=45 Jul 21 14:26:26.978817 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:26.978730 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:26:27.054311 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.054264 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75e95704-2138-4b63-a563-d08fbc924691-tls-certs\") pod \"75e95704-2138-4b63-a563-d08fbc924691\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " Jul 21 14:26:27.054311 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.054313 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-dshm\") pod \"75e95704-2138-4b63-a563-d08fbc924691\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " Jul 21 14:26:27.054583 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.054334 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-home\") pod \"75e95704-2138-4b63-a563-d08fbc924691\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " Jul 21 14:26:27.054583 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.054360 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-tmp-dir\") pod \"75e95704-2138-4b63-a563-d08fbc924691\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " Jul 21 14:26:27.054583 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.054396 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwfnx\" (UniqueName: \"kubernetes.io/projected/75e95704-2138-4b63-a563-d08fbc924691-kube-api-access-dwfnx\") pod \"75e95704-2138-4b63-a563-d08fbc924691\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " Jul 21 14:26:27.054583 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.054494 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-model-cache\") pod \"75e95704-2138-4b63-a563-d08fbc924691\" (UID: \"75e95704-2138-4b63-a563-d08fbc924691\") " Jul 21 14:26:27.054853 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.054633 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-home" (OuterVolumeSpecName: "home") pod "75e95704-2138-4b63-a563-d08fbc924691" (UID: "75e95704-2138-4b63-a563-d08fbc924691"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:26:27.054853 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.054676 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "75e95704-2138-4b63-a563-d08fbc924691" (UID: "75e95704-2138-4b63-a563-d08fbc924691"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:26:27.054961 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.054926 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-model-cache" (OuterVolumeSpecName: "model-cache") pod "75e95704-2138-4b63-a563-d08fbc924691" (UID: "75e95704-2138-4b63-a563-d08fbc924691"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:26:27.055012 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.054945 2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-home\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:26:27.055012 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.054978 2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-tmp-dir\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:26:27.057008 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.056970 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-dshm" (OuterVolumeSpecName: "dshm") pod "75e95704-2138-4b63-a563-d08fbc924691" (UID: "75e95704-2138-4b63-a563-d08fbc924691"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:26:27.057136 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.056985 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e95704-2138-4b63-a563-d08fbc924691-kube-api-access-dwfnx" (OuterVolumeSpecName: "kube-api-access-dwfnx") pod "75e95704-2138-4b63-a563-d08fbc924691" (UID: "75e95704-2138-4b63-a563-d08fbc924691"). InnerVolumeSpecName "kube-api-access-dwfnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:26:27.057136 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.057040 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75e95704-2138-4b63-a563-d08fbc924691-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "75e95704-2138-4b63-a563-d08fbc924691" (UID: "75e95704-2138-4b63-a563-d08fbc924691"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 14:26:27.155575 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.155528 2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75e95704-2138-4b63-a563-d08fbc924691-tls-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:26:27.155575 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.155566 2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-dshm\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:26:27.155575 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.155577 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwfnx\" (UniqueName: \"kubernetes.io/projected/75e95704-2138-4b63-a563-d08fbc924691-kube-api-access-dwfnx\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:26:27.155575 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.155586 2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75e95704-2138-4b63-a563-d08fbc924691-model-cache\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:26:27.516200 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.516162 2567 generic.go:356] "Generic (PLEG): container finished" podID="75e95704-2138-4b63-a563-d08fbc924691" containerID="3399b2af94c74780dbd88a8be2e2a690912fd541233cfe6898992924354bf7c2" exitCode=0 Jul 21 14:26:27.516390 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.516228 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" Jul 21 14:26:27.516390 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.516252 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" event={"ID":"75e95704-2138-4b63-a563-d08fbc924691","Type":"ContainerDied","Data":"3399b2af94c74780dbd88a8be2e2a690912fd541233cfe6898992924354bf7c2"} Jul 21 14:26:27.516390 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.516295 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr" event={"ID":"75e95704-2138-4b63-a563-d08fbc924691","Type":"ContainerDied","Data":"7656245b33be469a2c6192da4d85deac762023b00b7193c0c7829b2aacb868f2"} Jul 21 14:26:27.516390 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.516314 2567 scope.go:117] "RemoveContainer" containerID="3399b2af94c74780dbd88a8be2e2a690912fd541233cfe6898992924354bf7c2" Jul 21 14:26:27.527055 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.527030 2567 scope.go:117] "RemoveContainer" containerID="3399b2af94c74780dbd88a8be2e2a690912fd541233cfe6898992924354bf7c2" Jul 21 14:26:27.527468 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:26:27.527437 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3399b2af94c74780dbd88a8be2e2a690912fd541233cfe6898992924354bf7c2\": container with ID starting with 3399b2af94c74780dbd88a8be2e2a690912fd541233cfe6898992924354bf7c2 not found: ID does not exist" containerID="3399b2af94c74780dbd88a8be2e2a690912fd541233cfe6898992924354bf7c2" Jul 21 14:26:27.527583 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.527480 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3399b2af94c74780dbd88a8be2e2a690912fd541233cfe6898992924354bf7c2"} err="failed to get container status \"3399b2af94c74780dbd88a8be2e2a690912fd541233cfe6898992924354bf7c2\": rpc error: code = NotFound desc = could not find container \"3399b2af94c74780dbd88a8be2e2a690912fd541233cfe6898992924354bf7c2\": container with ID starting with 3399b2af94c74780dbd88a8be2e2a690912fd541233cfe6898992924354bf7c2 not found: ID does not exist" Jul 21 14:26:27.540643 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.540610 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr"] Jul 21 14:26:27.543014 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:27.542983 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-52967c/auth-test-model-52967c-kserve-d57bd85c9-2fbfr"] Jul 21 14:26:29.534057 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:29.534023 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75e95704-2138-4b63-a563-d08fbc924691" path="/var/lib/kubelet/pods/75e95704-2138-4b63-a563-d08fbc924691/volumes" Jul 21 14:26:43.954904 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:43.954857 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:26:45.854196 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:45.854152 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:26:47.601121 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:26:47.601073 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:27:35.838328 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:27:35.838243 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:27:36.709315 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:27:36.709283 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:27:53.113294 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:27:53.113244 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:28:05.590364 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:05.590320 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:28:25.519209 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:25.519173 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:28:25.616513 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:25.616476 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:28:30.384687 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:30.384649 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:28:30.488427 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:30.488391 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:28:32.106131 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.106076 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2"] Jul 21 14:28:32.106754 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.106733 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e95704-2138-4b63-a563-d08fbc924691" containerName="main" Jul 21 14:28:32.106857 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.106756 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e95704-2138-4b63-a563-d08fbc924691" containerName="main" Jul 21 14:28:32.106956 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.106942 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="75e95704-2138-4b63-a563-d08fbc924691" containerName="main" Jul 21 14:28:32.110389 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.110350 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.116217 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.116172 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-52967c\"/\"default-dockercfg-qfcs7\"" Jul 21 14:28:32.116217 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.116197 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-52967c\"/\"rate-test-model-52967c-kserve-self-signed-certs\"" Jul 21 14:28:32.116217 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.116215 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-52967c\"/\"kube-root-ca.crt\"" Jul 21 14:28:32.116529 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.116286 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-52967c\"/\"openshift-service-ca.crt\"" Jul 21 14:28:32.133402 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.133361 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a28dff88-321a-4a84-84f9-2c06851f4377-tls-certs\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.133614 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.133467 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-tmp-dir\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.133614 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.133508 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-model-cache\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.133614 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.133534 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-home\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.133614 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.133561 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xssnc\" (UniqueName: \"kubernetes.io/projected/a28dff88-321a-4a84-84f9-2c06851f4377-kube-api-access-xssnc\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.133614 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.133602 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-dshm\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.134489 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.134462 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2"] Jul 21 14:28:32.158579 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.158544 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:28:32.235289 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.235240 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a28dff88-321a-4a84-84f9-2c06851f4377-tls-certs\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.235533 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.235315 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-tmp-dir\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.235533 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.235340 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-model-cache\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.235533 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.235358 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-home\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.235533 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.235380 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xssnc\" (UniqueName: \"kubernetes.io/projected/a28dff88-321a-4a84-84f9-2c06851f4377-kube-api-access-xssnc\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.235533 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.235433 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-dshm\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.235920 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.235897 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-tmp-dir\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.235980 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.235915 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-home\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.235980 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.235955 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-model-cache\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.237924 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.237888 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-dshm\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.238043 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.238031 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a28dff88-321a-4a84-84f9-2c06851f4377-tls-certs\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.246221 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.246187 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xssnc\" (UniqueName: \"kubernetes.io/projected/a28dff88-321a-4a84-84f9-2c06851f4377-kube-api-access-xssnc\") pod \"rate-test-model-52967c-kserve-84d7d9c6cf-nqww2\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.421469 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.421404 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:32.594148 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:32.594100 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2"] Jul 21 14:28:32.596497 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:28:32.596455 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda28dff88_321a_4a84_84f9_2c06851f4377.slice/crio-9d2931c433d198c430499b2332d17681ef441f99835b9b150bef97555df2a968 WatchSource:0}: Error finding container 9d2931c433d198c430499b2332d17681ef441f99835b9b150bef97555df2a968: Status 404 returned error can't find the container with id 9d2931c433d198c430499b2332d17681ef441f99835b9b150bef97555df2a968 Jul 21 14:28:33.304129 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:33.304084 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" event={"ID":"a28dff88-321a-4a84-84f9-2c06851f4377","Type":"ContainerStarted","Data":"b25a9cf7c2ce88ecb6cf605cc198b1772fb7fd892a22d947c12b008b28bb01ff"} Jul 21 14:28:33.304129 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:33.304137 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" event={"ID":"a28dff88-321a-4a84-84f9-2c06851f4377","Type":"ContainerStarted","Data":"9d2931c433d198c430499b2332d17681ef441f99835b9b150bef97555df2a968"} Jul 21 14:28:33.325909 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:33.325819 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" podStartSLOduration=1.32580079 podStartE2EDuration="1.32580079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:28:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:28:33.321911673 +0000 UTC m=+2674.473752785" watchObservedRunningTime="2026-07-21 14:28:33.32580079 +0000 UTC m=+2674.477641898" Jul 21 14:28:33.744273 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:33.744231 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:28:33.841196 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:33.841159 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:28:42.422150 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:42.422103 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:42.422762 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:42.422170 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:42.435404 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:42.435360 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:43.375590 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:43.375556 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:28:59.620450 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:59.620365 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:28:59.630290 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:28:59.630260 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:29:32.491851 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:32.491803 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:29:33.918959 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:33.918908 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2"] Jul 21 14:29:33.926712 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:33.926662 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:29:48.940623 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:48.940565 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" podUID="a28dff88-321a-4a84-84f9-2c06851f4377" containerName="main" containerID="cri-o://b25a9cf7c2ce88ecb6cf605cc198b1772fb7fd892a22d947c12b008b28bb01ff" gracePeriod=45 Jul 21 14:29:49.220411 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.220375 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:29:49.346808 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.346769 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-tmp-dir\") pod \"a28dff88-321a-4a84-84f9-2c06851f4377\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " Jul 21 14:29:49.347004 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.346866 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-home\") pod \"a28dff88-321a-4a84-84f9-2c06851f4377\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " Jul 21 14:29:49.347004 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.346943 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-dshm\") pod \"a28dff88-321a-4a84-84f9-2c06851f4377\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " Jul 21 14:29:49.347004 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.346983 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-model-cache\") pod \"a28dff88-321a-4a84-84f9-2c06851f4377\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " Jul 21 14:29:49.347181 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.347060 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a28dff88-321a-4a84-84f9-2c06851f4377-tls-certs\") pod \"a28dff88-321a-4a84-84f9-2c06851f4377\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " Jul 21 14:29:49.347181 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.347096 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xssnc\" (UniqueName: \"kubernetes.io/projected/a28dff88-321a-4a84-84f9-2c06851f4377-kube-api-access-xssnc\") pod \"a28dff88-321a-4a84-84f9-2c06851f4377\" (UID: \"a28dff88-321a-4a84-84f9-2c06851f4377\") " Jul 21 14:29:49.347181 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.347097 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a28dff88-321a-4a84-84f9-2c06851f4377" (UID: "a28dff88-321a-4a84-84f9-2c06851f4377"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:29:49.347181 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.347111 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-home" (OuterVolumeSpecName: "home") pod "a28dff88-321a-4a84-84f9-2c06851f4377" (UID: "a28dff88-321a-4a84-84f9-2c06851f4377"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:29:49.347414 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.347345 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-model-cache" (OuterVolumeSpecName: "model-cache") pod "a28dff88-321a-4a84-84f9-2c06851f4377" (UID: "a28dff88-321a-4a84-84f9-2c06851f4377"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:29:49.347514 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.347489 2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-home\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:29:49.347573 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.347514 2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-model-cache\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:29:49.347573 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.347531 2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-tmp-dir\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:29:49.349360 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.349316 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-dshm" (OuterVolumeSpecName: "dshm") pod "a28dff88-321a-4a84-84f9-2c06851f4377" (UID: "a28dff88-321a-4a84-84f9-2c06851f4377"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:29:49.349514 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.349418 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a28dff88-321a-4a84-84f9-2c06851f4377-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a28dff88-321a-4a84-84f9-2c06851f4377" (UID: "a28dff88-321a-4a84-84f9-2c06851f4377"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 14:29:49.349640 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.349619 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a28dff88-321a-4a84-84f9-2c06851f4377-kube-api-access-xssnc" (OuterVolumeSpecName: "kube-api-access-xssnc") pod "a28dff88-321a-4a84-84f9-2c06851f4377" (UID: "a28dff88-321a-4a84-84f9-2c06851f4377"). InnerVolumeSpecName "kube-api-access-xssnc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:29:49.448600 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.448498 2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a28dff88-321a-4a84-84f9-2c06851f4377-dshm\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:29:49.448600 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.448531 2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a28dff88-321a-4a84-84f9-2c06851f4377-tls-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:29:49.448600 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.448544 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xssnc\" (UniqueName: \"kubernetes.io/projected/a28dff88-321a-4a84-84f9-2c06851f4377-kube-api-access-xssnc\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:29:49.779226 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.779123 2567 generic.go:356] "Generic (PLEG): container finished" podID="a28dff88-321a-4a84-84f9-2c06851f4377" containerID="b25a9cf7c2ce88ecb6cf605cc198b1772fb7fd892a22d947c12b008b28bb01ff" exitCode=0 Jul 21 14:29:49.779226 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.779168 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" Jul 21 14:29:49.779226 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.779197 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" event={"ID":"a28dff88-321a-4a84-84f9-2c06851f4377","Type":"ContainerDied","Data":"b25a9cf7c2ce88ecb6cf605cc198b1772fb7fd892a22d947c12b008b28bb01ff"} Jul 21 14:29:49.779509 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.779244 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2" event={"ID":"a28dff88-321a-4a84-84f9-2c06851f4377","Type":"ContainerDied","Data":"9d2931c433d198c430499b2332d17681ef441f99835b9b150bef97555df2a968"} Jul 21 14:29:49.779509 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.779281 2567 scope.go:117] "RemoveContainer" containerID="b25a9cf7c2ce88ecb6cf605cc198b1772fb7fd892a22d947c12b008b28bb01ff" Jul 21 14:29:49.789179 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.789158 2567 scope.go:117] "RemoveContainer" containerID="b25a9cf7c2ce88ecb6cf605cc198b1772fb7fd892a22d947c12b008b28bb01ff" Jul 21 14:29:49.789555 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:29:49.789535 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b25a9cf7c2ce88ecb6cf605cc198b1772fb7fd892a22d947c12b008b28bb01ff\": container with ID starting with b25a9cf7c2ce88ecb6cf605cc198b1772fb7fd892a22d947c12b008b28bb01ff not found: ID does not exist" containerID="b25a9cf7c2ce88ecb6cf605cc198b1772fb7fd892a22d947c12b008b28bb01ff" Jul 21 14:29:49.789615 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.789570 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b25a9cf7c2ce88ecb6cf605cc198b1772fb7fd892a22d947c12b008b28bb01ff"} err="failed to get container status \"b25a9cf7c2ce88ecb6cf605cc198b1772fb7fd892a22d947c12b008b28bb01ff\": rpc error: code = NotFound desc = could not find container \"b25a9cf7c2ce88ecb6cf605cc198b1772fb7fd892a22d947c12b008b28bb01ff\": container with ID starting with b25a9cf7c2ce88ecb6cf605cc198b1772fb7fd892a22d947c12b008b28bb01ff not found: ID does not exist" Jul 21 14:29:49.808137 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.808095 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2"] Jul 21 14:29:49.817592 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:49.817551 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-52967c/rate-test-model-52967c-kserve-84d7d9c6cf-nqww2"] Jul 21 14:29:51.534218 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:51.534177 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a28dff88-321a-4a84-84f9-2c06851f4377" path="/var/lib/kubelet/pods/a28dff88-321a-4a84-84f9-2c06851f4377/volumes" Jul 21 14:29:53.167975 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:53.167936 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:29:53.268717 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:29:53.268664 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:03.207094 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:03.207036 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:07.727761 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:07.727690 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:08.302643 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:08.302605 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:14.974558 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:14.974480 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:15.047227 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:15.047187 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:15.234906 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:15.234611 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx"] Jul 21 14:30:15.235290 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:15.235231 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerName="istio-proxy" containerID="cri-o://0467424b7093b5693067157f616ce9a5207d7ee121d1ab14f62ce8ed2630d01b" gracePeriod=30 Jul 21 14:30:15.252251 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:15.252189 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:15.473570 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:15.473534 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:16.148030 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:16.147987 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:16.249246 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:16.249199 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:20.529038 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.528990 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:30:20.583406 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.583346 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-data\") pod \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " Jul 21 14:30:20.583406 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.583408 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istiod-ca-cert\") pod \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " Jul 21 14:30:20.583673 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.583440 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-workload-certs\") pod \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " Jul 21 14:30:20.583673 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.583490 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-workload-socket\") pod \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " Jul 21 14:30:20.583673 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.583519 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-envoy\") pod \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " Jul 21 14:30:20.583673 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.583563 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-podinfo\") pod \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " Jul 21 14:30:20.583673 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.583598 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-token\") pod \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " Jul 21 14:30:20.583673 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.583625 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8hjb\" (UniqueName: \"kubernetes.io/projected/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-kube-api-access-g8hjb\") pod \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " Jul 21 14:30:20.584063 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.583722 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-credential-socket\") pod \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\" (UID: \"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b\") " Jul 21 14:30:20.584063 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.583879 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" (UID: "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:30:20.584063 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.583888 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" (UID: "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:30:20.584063 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.583958 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" (UID: "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 14:30:20.584285 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.584260 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" (UID: "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:30:20.585414 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.584857 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-data" (OuterVolumeSpecName: "istio-data") pod "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" (UID: "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:30:20.585414 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.585257 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istiod-ca-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:20.585414 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.585281 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-workload-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:20.585414 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.585294 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-workload-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:20.586362 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.586336 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-token" (OuterVolumeSpecName: "istio-token") pod "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" (UID: "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:30:20.586522 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.586490 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" (UID: "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:30:20.586632 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.586571 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-kube-api-access-g8hjb" (OuterVolumeSpecName: "kube-api-access-g8hjb") pod "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" (UID: "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b"). InnerVolumeSpecName "kube-api-access-g8hjb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:30:20.586679 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.586657 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" (UID: "3e75e1d0-15a7-4372-8fe8-40dc19bcc06b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 14:30:20.686260 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.686200 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-credential-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:20.686260 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.686251 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-data\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:20.686260 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.686262 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-envoy\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:20.686260 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.686271 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-podinfo\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:20.686260 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.686280 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-istio-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:20.686602 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.686289 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8hjb\" (UniqueName: \"kubernetes.io/projected/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b-kube-api-access-g8hjb\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:20.978021 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.977953 2567 generic.go:356] "Generic (PLEG): container finished" podID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerID="0467424b7093b5693067157f616ce9a5207d7ee121d1ab14f62ce8ed2630d01b" exitCode=0 Jul 21 14:30:20.978439 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.978049 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" Jul 21 14:30:20.978439 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.978042 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" event={"ID":"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b","Type":"ContainerDied","Data":"0467424b7093b5693067157f616ce9a5207d7ee121d1ab14f62ce8ed2630d01b"} Jul 21 14:30:20.978439 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.978142 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx" event={"ID":"3e75e1d0-15a7-4372-8fe8-40dc19bcc06b","Type":"ContainerDied","Data":"7515f31a7026bd8cd7f6b4f45314381832d6d562d253592097e33aea6a5160ca"} Jul 21 14:30:20.978439 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.978175 2567 scope.go:117] "RemoveContainer" containerID="0467424b7093b5693067157f616ce9a5207d7ee121d1ab14f62ce8ed2630d01b" Jul 21 14:30:20.996672 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:20.996639 2567 scope.go:117] "RemoveContainer" containerID="b1c857159a390e20f018b0175391f1ad489f5201885b78b945c23f9dfd51d33c" Jul 21 14:30:21.010589 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:21.010451 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx"] Jul 21 14:30:21.015745 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:21.015439 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-67be07-openshift-default-74b44f547f-9pcmx"] Jul 21 14:30:21.019368 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:21.018756 2567 scope.go:117] "RemoveContainer" containerID="0467424b7093b5693067157f616ce9a5207d7ee121d1ab14f62ce8ed2630d01b" Jul 21 14:30:21.020176 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:30:21.020115 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0467424b7093b5693067157f616ce9a5207d7ee121d1ab14f62ce8ed2630d01b\": container with ID starting with 0467424b7093b5693067157f616ce9a5207d7ee121d1ab14f62ce8ed2630d01b not found: ID does not exist" containerID="0467424b7093b5693067157f616ce9a5207d7ee121d1ab14f62ce8ed2630d01b" Jul 21 14:30:21.020379 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:21.020212 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0467424b7093b5693067157f616ce9a5207d7ee121d1ab14f62ce8ed2630d01b"} err="failed to get container status \"0467424b7093b5693067157f616ce9a5207d7ee121d1ab14f62ce8ed2630d01b\": rpc error: code = NotFound desc = could not find container \"0467424b7093b5693067157f616ce9a5207d7ee121d1ab14f62ce8ed2630d01b\": container with ID starting with 0467424b7093b5693067157f616ce9a5207d7ee121d1ab14f62ce8ed2630d01b not found: ID does not exist" Jul 21 14:30:21.020379 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:21.020267 2567 scope.go:117] "RemoveContainer" containerID="b1c857159a390e20f018b0175391f1ad489f5201885b78b945c23f9dfd51d33c" Jul 21 14:30:21.020883 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:30:21.020851 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1c857159a390e20f018b0175391f1ad489f5201885b78b945c23f9dfd51d33c\": container with ID starting with b1c857159a390e20f018b0175391f1ad489f5201885b78b945c23f9dfd51d33c not found: ID does not exist" containerID="b1c857159a390e20f018b0175391f1ad489f5201885b78b945c23f9dfd51d33c" Jul 21 14:30:21.021017 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:21.020896 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1c857159a390e20f018b0175391f1ad489f5201885b78b945c23f9dfd51d33c"} err="failed to get container status \"b1c857159a390e20f018b0175391f1ad489f5201885b78b945c23f9dfd51d33c\": rpc error: code = NotFound desc = could not find container \"b1c857159a390e20f018b0175391f1ad489f5201885b78b945c23f9dfd51d33c\": container with ID starting with b1c857159a390e20f018b0175391f1ad489f5201885b78b945c23f9dfd51d33c not found: ID does not exist" Jul 21 14:30:21.533730 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:21.533591 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" path="/var/lib/kubelet/pods/3e75e1d0-15a7-4372-8fe8-40dc19bcc06b/volumes" Jul 21 14:30:26.152426 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:26.152381 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:30.674287 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:30.674243 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:31.292992 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:31.292917 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:37.553662 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:37.553627 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:37.655014 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:37.654976 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:37.818256 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:37.818162 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn"] Jul 21 14:30:37.818465 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:37.818423 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" containerID="cri-o://d2b13be3f98f993acd08dc86afa64a4e692bfd3fbdc6a0d0dc51251a656af2e1" gracePeriod=30 Jul 21 14:30:37.850537 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:37.850493 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:38.074222 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:38.074107 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:38.470506 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:38.470461 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78bf958d9-8ndr2_cf8adfce-4ec2-4552-8c9d-16bdeb50dfab/authorino/0.log" Jul 21 14:30:43.100097 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.100062 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:30:43.117872 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.117837 2567 generic.go:356] "Generic (PLEG): container finished" podID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerID="d2b13be3f98f993acd08dc86afa64a4e692bfd3fbdc6a0d0dc51251a656af2e1" exitCode=0 Jul 21 14:30:43.118070 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.117889 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" Jul 21 14:30:43.118070 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.117920 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" event={"ID":"08375ad4-b016-4f95-af61-bdaa2b07c490","Type":"ContainerDied","Data":"d2b13be3f98f993acd08dc86afa64a4e692bfd3fbdc6a0d0dc51251a656af2e1"} Jul 21 14:30:43.118070 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.117994 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn" event={"ID":"08375ad4-b016-4f95-af61-bdaa2b07c490","Type":"ContainerDied","Data":"470696633ae56fa3e5bc492392ae656cc70090251aa65e0fcd310e43a1e0f4ce"} Jul 21 14:30:43.118070 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.118021 2567 scope.go:117] "RemoveContainer" containerID="d2b13be3f98f993acd08dc86afa64a4e692bfd3fbdc6a0d0dc51251a656af2e1" Jul 21 14:30:43.128595 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.128574 2567 scope.go:117] "RemoveContainer" containerID="7eeee1b4a5422ad6a2fb2dbac7e86a3ae0efe344c9f247714fbb3e4bbd921e47" Jul 21 14:30:43.138218 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.138197 2567 scope.go:117] "RemoveContainer" containerID="d2b13be3f98f993acd08dc86afa64a4e692bfd3fbdc6a0d0dc51251a656af2e1" Jul 21 14:30:43.138581 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:30:43.138559 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2b13be3f98f993acd08dc86afa64a4e692bfd3fbdc6a0d0dc51251a656af2e1\": container with ID starting with d2b13be3f98f993acd08dc86afa64a4e692bfd3fbdc6a0d0dc51251a656af2e1 not found: ID does not exist" containerID="d2b13be3f98f993acd08dc86afa64a4e692bfd3fbdc6a0d0dc51251a656af2e1" Jul 21 14:30:43.138657 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.138593 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2b13be3f98f993acd08dc86afa64a4e692bfd3fbdc6a0d0dc51251a656af2e1"} err="failed to get container status \"d2b13be3f98f993acd08dc86afa64a4e692bfd3fbdc6a0d0dc51251a656af2e1\": rpc error: code = NotFound desc = could not find container \"d2b13be3f98f993acd08dc86afa64a4e692bfd3fbdc6a0d0dc51251a656af2e1\": container with ID starting with d2b13be3f98f993acd08dc86afa64a4e692bfd3fbdc6a0d0dc51251a656af2e1 not found: ID does not exist" Jul 21 14:30:43.138657 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.138613 2567 scope.go:117] "RemoveContainer" containerID="7eeee1b4a5422ad6a2fb2dbac7e86a3ae0efe344c9f247714fbb3e4bbd921e47" Jul 21 14:30:43.138965 ip-10-0-129-212 kubenswrapper[2567]: E0721 14:30:43.138945 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7eeee1b4a5422ad6a2fb2dbac7e86a3ae0efe344c9f247714fbb3e4bbd921e47\": container with ID starting with 7eeee1b4a5422ad6a2fb2dbac7e86a3ae0efe344c9f247714fbb3e4bbd921e47 not found: ID does not exist" containerID="7eeee1b4a5422ad6a2fb2dbac7e86a3ae0efe344c9f247714fbb3e4bbd921e47" Jul 21 14:30:43.139029 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.138976 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eeee1b4a5422ad6a2fb2dbac7e86a3ae0efe344c9f247714fbb3e4bbd921e47"} err="failed to get container status \"7eeee1b4a5422ad6a2fb2dbac7e86a3ae0efe344c9f247714fbb3e4bbd921e47\": rpc error: code = NotFound desc = could not find container \"7eeee1b4a5422ad6a2fb2dbac7e86a3ae0efe344c9f247714fbb3e4bbd921e47\": container with ID starting with 7eeee1b4a5422ad6a2fb2dbac7e86a3ae0efe344c9f247714fbb3e4bbd921e47 not found: ID does not exist" Jul 21 14:30:43.217210 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.217164 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08375ad4-b016-4f95-af61-bdaa2b07c490-istiod-ca-cert\") pod \"08375ad4-b016-4f95-af61-bdaa2b07c490\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " Jul 21 14:30:43.217433 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.217264 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-envoy\") pod \"08375ad4-b016-4f95-af61-bdaa2b07c490\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " Jul 21 14:30:43.217433 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.217287 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-credential-socket\") pod \"08375ad4-b016-4f95-af61-bdaa2b07c490\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " Jul 21 14:30:43.217433 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.217307 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2hwr\" (UniqueName: \"kubernetes.io/projected/08375ad4-b016-4f95-af61-bdaa2b07c490-kube-api-access-p2hwr\") pod \"08375ad4-b016-4f95-af61-bdaa2b07c490\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " Jul 21 14:30:43.217433 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.217336 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-workload-certs\") pod \"08375ad4-b016-4f95-af61-bdaa2b07c490\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " Jul 21 14:30:43.217433 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.217361 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-data\") pod \"08375ad4-b016-4f95-af61-bdaa2b07c490\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " Jul 21 14:30:43.217433 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.217383 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-token\") pod \"08375ad4-b016-4f95-af61-bdaa2b07c490\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " Jul 21 14:30:43.217433 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.217419 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-workload-socket\") pod \"08375ad4-b016-4f95-af61-bdaa2b07c490\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " Jul 21 14:30:43.217853 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.217468 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-podinfo\") pod \"08375ad4-b016-4f95-af61-bdaa2b07c490\" (UID: \"08375ad4-b016-4f95-af61-bdaa2b07c490\") " Jul 21 14:30:43.217853 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.217731 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "08375ad4-b016-4f95-af61-bdaa2b07c490" (UID: "08375ad4-b016-4f95-af61-bdaa2b07c490"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:30:43.217853 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.217673 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08375ad4-b016-4f95-af61-bdaa2b07c490-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "08375ad4-b016-4f95-af61-bdaa2b07c490" (UID: "08375ad4-b016-4f95-af61-bdaa2b07c490"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 14:30:43.218022 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.217882 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-workload-certs\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:43.218072 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.218016 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "08375ad4-b016-4f95-af61-bdaa2b07c490" (UID: "08375ad4-b016-4f95-af61-bdaa2b07c490"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:30:43.218108 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.218061 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "08375ad4-b016-4f95-af61-bdaa2b07c490" (UID: "08375ad4-b016-4f95-af61-bdaa2b07c490"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:30:43.218569 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.218532 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-data" (OuterVolumeSpecName: "istio-data") pod "08375ad4-b016-4f95-af61-bdaa2b07c490" (UID: "08375ad4-b016-4f95-af61-bdaa2b07c490"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:30:43.219894 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.219869 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "08375ad4-b016-4f95-af61-bdaa2b07c490" (UID: "08375ad4-b016-4f95-af61-bdaa2b07c490"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 14:30:43.219998 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.219886 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08375ad4-b016-4f95-af61-bdaa2b07c490-kube-api-access-p2hwr" (OuterVolumeSpecName: "kube-api-access-p2hwr") pod "08375ad4-b016-4f95-af61-bdaa2b07c490" (UID: "08375ad4-b016-4f95-af61-bdaa2b07c490"). InnerVolumeSpecName "kube-api-access-p2hwr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:30:43.220174 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.220153 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "08375ad4-b016-4f95-af61-bdaa2b07c490" (UID: "08375ad4-b016-4f95-af61-bdaa2b07c490"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 14:30:43.220231 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.220152 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-token" (OuterVolumeSpecName: "istio-token") pod "08375ad4-b016-4f95-af61-bdaa2b07c490" (UID: "08375ad4-b016-4f95-af61-bdaa2b07c490"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 14:30:43.319444 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.319396 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-podinfo\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:43.319444 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.319432 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08375ad4-b016-4f95-af61-bdaa2b07c490-istiod-ca-cert\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:43.319444 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.319447 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-envoy\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:43.319744 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.319460 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-credential-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:43.319744 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.319469 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2hwr\" (UniqueName: \"kubernetes.io/projected/08375ad4-b016-4f95-af61-bdaa2b07c490-kube-api-access-p2hwr\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:43.319744 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.319478 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-data\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:43.319744 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.319486 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08375ad4-b016-4f95-af61-bdaa2b07c490-istio-token\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:43.319744 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.319493 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08375ad4-b016-4f95-af61-bdaa2b07c490-workload-socket\") on node \"ip-10-0-129-212.ec2.internal\" DevicePath \"\"" Jul 21 14:30:43.443106 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.443063 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn"] Jul 21 14:30:43.451794 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.451743 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-52967c-openshift-default-5cf75bc8c9-npzsn"] Jul 21 14:30:43.533218 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:43.533175 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" path="/var/lib/kubelet/pods/08375ad4-b016-4f95-af61-bdaa2b07c490/volumes" Jul 21 14:30:44.509583 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:44.509549 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d98d8798d-jmhl7_150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6/copy-manifests/0.log" Jul 21 14:30:44.517005 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:44.516973 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d98d8798d-jmhl7_150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6/manager/0.log" Jul 21 14:30:46.192037 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.192000 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv_103cc13a-a074-47bf-aeac-68deb81f3520/extract/0.log" Jul 21 14:30:46.198458 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.198430 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv_103cc13a-a074-47bf-aeac-68deb81f3520/util/0.log" Jul 21 14:30:46.205808 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.205786 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv_103cc13a-a074-47bf-aeac-68deb81f3520/pull/0.log" Jul 21 14:30:46.316954 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.316920 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6_7d6bb23e-1c8c-4bc4-8edf-dc27214e9631/util/0.log" Jul 21 14:30:46.323910 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.323871 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6_7d6bb23e-1c8c-4bc4-8edf-dc27214e9631/pull/0.log" Jul 21 14:30:46.331644 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.331612 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6_7d6bb23e-1c8c-4bc4-8edf-dc27214e9631/extract/0.log" Jul 21 14:30:46.441859 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.441821 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb_603b82ea-622a-461e-9114-4f787c689dbf/util/0.log" Jul 21 14:30:46.449172 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.449091 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb_603b82ea-622a-461e-9114-4f787c689dbf/pull/0.log" Jul 21 14:30:46.456444 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.456418 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb_603b82ea-622a-461e-9114-4f787c689dbf/extract/0.log" Jul 21 14:30:46.573415 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.573351 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9_3b983d46-27cc-422c-9285-55da67e6c155/util/0.log" Jul 21 14:30:46.581725 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.581646 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9_3b983d46-27cc-422c-9285-55da67e6c155/pull/0.log" Jul 21 14:30:46.590280 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.590220 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9_3b983d46-27cc-422c-9285-55da67e6c155/extract/0.log" Jul 21 14:30:46.715361 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.715254 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78bf958d9-8ndr2_cf8adfce-4ec2-4552-8c9d-16bdeb50dfab/authorino/0.log" Jul 21 14:30:46.940338 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:46.940304 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-d4qgk_5110a6b1-b898-43c4-8229-62dcb157f267/manager/0.log" Jul 21 14:30:47.180963 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:47.180913 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4ktmm_e8edc027-509d-4241-91c1-0f3caf4f5a67/registry-server/0.log" Jul 21 14:30:47.300422 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:47.300380 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-dntz8_cb1c7b36-935b-4a47-9e77-2b394a660b05/manager/0.log" Jul 21 14:30:47.522304 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:47.522216 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jpbms_5f4b4477-39a1-4f98-9d08-d70bcffb489a/manager/0.log" Jul 21 14:30:47.860168 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:47.860076 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-554689689d996fx_6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5/istio-proxy/0.log" Jul 21 14:30:48.318750 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:48.318711 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74949dcd89-dlm5x_5a886230-716b-4ef6-97ae-0847508d4ab7/istio-proxy/1.log" Jul 21 14:30:48.562337 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:48.562302 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-f45qg_f668a1ed-b892-4b42-9151-66990c6511e4/payload-processing/0.log" Jul 21 14:30:48.675066 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:48.675035 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5565c7c56b-9stjh_d29c3f6d-9285-48fa-979b-b71b91753fc1/router/0.log" Jul 21 14:30:49.237375 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:49.237340 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-f2bnr_0f0685b3-e171-4e33-ab1e-bf64414d5e68/main/0.log" Jul 21 14:30:49.588980 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:49.588887 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-c2ktp_c40bffa9-57b8-4427-8aa7-96ac6a4e8557/main/0.log" Jul 21 14:30:55.365492 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.365446 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fv4cl/must-gather-62ksm"] Jul 21 14:30:55.365936 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.365920 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerName="istio-proxy" Jul 21 14:30:55.365936 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.365933 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerName="istio-proxy" Jul 21 14:30:55.366020 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.365945 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a28dff88-321a-4a84-84f9-2c06851f4377" containerName="main" Jul 21 14:30:55.366020 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.365951 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a28dff88-321a-4a84-84f9-2c06851f4377" containerName="main" Jul 21 14:30:55.366020 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.365974 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" Jul 21 14:30:55.366020 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.365980 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" Jul 21 14:30:55.366020 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.365998 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" Jul 21 14:30:55.366020 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.366003 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" Jul 21 14:30:55.366200 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.366067 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" Jul 21 14:30:55.366200 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.366078 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerName="istio-proxy" Jul 21 14:30:55.366200 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.366085 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerName="istio-proxy" Jul 21 14:30:55.366200 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.366092 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a28dff88-321a-4a84-84f9-2c06851f4377" containerName="main" Jul 21 14:30:55.366200 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.366147 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerName="istio-proxy" Jul 21 14:30:55.366200 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.366153 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e75e1d0-15a7-4372-8fe8-40dc19bcc06b" containerName="istio-proxy" Jul 21 14:30:55.366397 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.366222 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="08375ad4-b016-4f95-af61-bdaa2b07c490" containerName="istio-proxy" Jul 21 14:30:55.371047 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.371020 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fv4cl/must-gather-62ksm" Jul 21 14:30:55.373888 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.373843 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fv4cl\"/\"kube-root-ca.crt\"" Jul 21 14:30:55.375219 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.375193 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fv4cl\"/\"default-dockercfg-zm8bc\"" Jul 21 14:30:55.375332 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.375193 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fv4cl\"/\"openshift-service-ca.crt\"" Jul 21 14:30:55.382048 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.381973 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fv4cl/must-gather-62ksm"] Jul 21 14:30:55.427558 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.427515 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:55.427787 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.427611 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dztzf\" (UniqueName: \"kubernetes.io/projected/3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217-kube-api-access-dztzf\") pod \"must-gather-62ksm\" (UID: \"3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217\") " pod="openshift-must-gather-fv4cl/must-gather-62ksm" Jul 21 14:30:55.427787 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.427747 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217-must-gather-output\") pod \"must-gather-62ksm\" (UID: \"3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217\") " pod="openshift-must-gather-fv4cl/must-gather-62ksm" Jul 21 14:30:55.529094 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.529057 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dztzf\" (UniqueName: \"kubernetes.io/projected/3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217-kube-api-access-dztzf\") pod \"must-gather-62ksm\" (UID: \"3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217\") " pod="openshift-must-gather-fv4cl/must-gather-62ksm" Jul 21 14:30:55.529314 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.529143 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217-must-gather-output\") pod \"must-gather-62ksm\" (UID: \"3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217\") " pod="openshift-must-gather-fv4cl/must-gather-62ksm" Jul 21 14:30:55.529553 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.529533 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217-must-gather-output\") pod \"must-gather-62ksm\" (UID: \"3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217\") " pod="openshift-must-gather-fv4cl/must-gather-62ksm" Jul 21 14:30:55.541211 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.541172 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" limit=1024 Jul 21 14:30:55.544086 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.544054 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dztzf\" (UniqueName: \"kubernetes.io/projected/3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217-kube-api-access-dztzf\") pod \"must-gather-62ksm\" (UID: \"3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217\") " pod="openshift-must-gather-fv4cl/must-gather-62ksm" Jul 21 14:30:55.683321 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.683274 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fv4cl/must-gather-62ksm" Jul 21 14:30:55.843731 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.843627 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fv4cl/must-gather-62ksm"] Jul 21 14:30:55.846758 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:30:55.846727 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ba447f2_52e4_42e5_bc4f_2e5ae1b1e217.slice/crio-4ebdf416033f46b586cac566c70ea2a5752d9247b1d6b83a59a9a9407ea4d003 WatchSource:0}: Error finding container 4ebdf416033f46b586cac566c70ea2a5752d9247b1d6b83a59a9a9407ea4d003: Status 404 returned error can't find the container with id 4ebdf416033f46b586cac566c70ea2a5752d9247b1d6b83a59a9a9407ea4d003 Jul 21 14:30:55.848858 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:55.848833 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 14:30:56.194431 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:56.194388 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fv4cl/must-gather-62ksm" event={"ID":"3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217","Type":"ContainerStarted","Data":"4ebdf416033f46b586cac566c70ea2a5752d9247b1d6b83a59a9a9407ea4d003"} Jul 21 14:30:58.210775 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:58.210736 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fv4cl/must-gather-62ksm" event={"ID":"3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217","Type":"ContainerStarted","Data":"b61440373bcec0f8a288db4bf00adfa0c0861b27deb6b027166d6e53963c448c"} Jul 21 14:30:58.210775 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:58.210784 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fv4cl/must-gather-62ksm" event={"ID":"3ba447f2-52e4-42e5-bc4f-2e5ae1b1e217","Type":"ContainerStarted","Data":"517e56266ca69a7146474fbf92a634a286c84ffbaebcab92fa4a423d9d843b10"} Jul 21 14:30:58.232359 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:58.232284 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fv4cl/must-gather-62ksm" podStartSLOduration=1.85367064 podStartE2EDuration="3.232255634s" totalImagesPullingTime="1.378584994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:30:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:30:58.230639998 +0000 UTC m=+2819.382481108" watchObservedRunningTime="2026-07-21 14:30:58.232255634 +0000 UTC m=+2819.384096744" Jul 21 14:30:59.119251 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:59.119212 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xvrvl_093b3a4b-ec79-4a4d-a86f-05ea62a88e2c/global-pull-secret-syncer/0.log" Jul 21 14:30:59.229085 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:59.229049 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5vvn4_67843201-85e5-47e7-8100-c1bdecbefcfe/konnectivity-agent/0.log" Jul 21 14:30:59.320255 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:30:59.320225 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-212.ec2.internal_6ccec470417826b6e67569d0c3ca8973/haproxy/0.log" Jul 21 14:31:04.581539 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:04.581495 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv_103cc13a-a074-47bf-aeac-68deb81f3520/extract/0.log" Jul 21 14:31:04.610804 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:04.610757 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv_103cc13a-a074-47bf-aeac-68deb81f3520/util/0.log" Jul 21 14:31:04.637998 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:04.637958 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759hh2hv_103cc13a-a074-47bf-aeac-68deb81f3520/pull/0.log" Jul 21 14:31:04.681073 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:04.681037 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6_7d6bb23e-1c8c-4bc4-8edf-dc27214e9631/extract/0.log" Jul 21 14:31:04.707653 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:04.707607 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6_7d6bb23e-1c8c-4bc4-8edf-dc27214e9631/util/0.log" Jul 21 14:31:04.739850 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:04.739815 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kgmt6_7d6bb23e-1c8c-4bc4-8edf-dc27214e9631/pull/0.log" Jul 21 14:31:04.777856 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:04.777828 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb_603b82ea-622a-461e-9114-4f787c689dbf/extract/0.log" Jul 21 14:31:04.802129 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:04.802081 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb_603b82ea-622a-461e-9114-4f787c689dbf/util/0.log" Jul 21 14:31:04.828779 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:04.828726 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xkjrb_603b82ea-622a-461e-9114-4f787c689dbf/pull/0.log" Jul 21 14:31:04.874692 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:04.874614 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9_3b983d46-27cc-422c-9285-55da67e6c155/extract/0.log" Jul 21 14:31:04.902148 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:04.902110 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9_3b983d46-27cc-422c-9285-55da67e6c155/util/0.log" Jul 21 14:31:04.928267 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:04.928237 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nqqf9_3b983d46-27cc-422c-9285-55da67e6c155/pull/0.log" Jul 21 14:31:05.467820 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:05.467784 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78bf958d9-8ndr2_cf8adfce-4ec2-4552-8c9d-16bdeb50dfab/authorino/0.log" Jul 21 14:31:05.524389 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:05.524357 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-d4qgk_5110a6b1-b898-43c4-8229-62dcb157f267/manager/0.log" Jul 21 14:31:05.636985 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:05.636948 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4ktmm_e8edc027-509d-4241-91c1-0f3caf4f5a67/registry-server/0.log" Jul 21 14:31:05.838713 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:05.838579 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-dntz8_cb1c7b36-935b-4a47-9e77-2b394a660b05/manager/0.log" Jul 21 14:31:05.963161 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:05.960930 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jpbms_5f4b4477-39a1-4f98-9d08-d70bcffb489a/manager/0.log" Jul 21 14:31:07.961301 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:07.961250 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-ng6jq_37b1bb9e-3013-4c9a-87b8-bc120b0a9f3a/cluster-monitoring-operator/0.log" Jul 21 14:31:08.147131 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:08.147089 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6zpbn_5a9e8cac-01bb-4fd0-9f41-3e8512e23572/node-exporter/0.log" Jul 21 14:31:08.173300 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:08.173266 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6zpbn_5a9e8cac-01bb-4fd0-9f41-3e8512e23572/kube-rbac-proxy/0.log" Jul 21 14:31:08.199540 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:08.199416 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6zpbn_5a9e8cac-01bb-4fd0-9f41-3e8512e23572/init-textfile/0.log" Jul 21 14:31:10.002936 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.002879 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf"] Jul 21 14:31:10.008246 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.008201 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.018633 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.018579 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf"] Jul 21 14:31:10.109331 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.109285 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-proc\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.109798 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.109772 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-sys\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.109996 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.109981 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-podres\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.110140 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.110123 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-lib-modules\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.110321 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.110302 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tp92\" (UniqueName: \"kubernetes.io/projected/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-kube-api-access-9tp92\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.211944 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.211890 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-sys\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.211944 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.211948 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-podres\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.212244 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.211971 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-lib-modules\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.212244 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.212003 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tp92\" (UniqueName: \"kubernetes.io/projected/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-kube-api-access-9tp92\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.212244 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.212023 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-proc\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.212244 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.212041 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-sys\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.212244 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.212188 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-podres\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.212244 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.212228 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-proc\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.212607 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.212297 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-lib-modules\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.228731 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.223329 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tp92\" (UniqueName: \"kubernetes.io/projected/a5be571b-53ce-44fd-9ae4-aa4cb7e45383-kube-api-access-9tp92\") pod \"perf-node-gather-daemonset-4mdwf\" (UID: \"a5be571b-53ce-44fd-9ae4-aa4cb7e45383\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.312216 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.312112 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-kfq8l_6b173759-de6d-43c8-adaa-41a0df593ed2/networking-console-plugin/0.log" Jul 21 14:31:10.324443 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.324418 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:10.556762 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:10.556731 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf"] Jul 21 14:31:10.559379 ip-10-0-129-212 kubenswrapper[2567]: W0721 14:31:10.559317 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda5be571b_53ce_44fd_9ae4_aa4cb7e45383.slice/crio-4c60d39198d8c8d3268d8cc965c0240b79b13b2170797e908ef8892f0ecd0e68 WatchSource:0}: Error finding container 4c60d39198d8c8d3268d8cc965c0240b79b13b2170797e908ef8892f0ecd0e68: Status 404 returned error can't find the container with id 4c60d39198d8c8d3268d8cc965c0240b79b13b2170797e908ef8892f0ecd0e68 Jul 21 14:31:11.020566 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:11.020535 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/1.log" Jul 21 14:31:11.027450 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:11.027416 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pdjqz_cd83129f-d2e9-4efc-99be-4bc5af88e534/console-operator/2.log" Jul 21 14:31:11.332020 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:11.331900 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" event={"ID":"a5be571b-53ce-44fd-9ae4-aa4cb7e45383","Type":"ContainerStarted","Data":"b1b7e28991fcc744610fc683b51ae2a2f4b710355b471c397295e632a2e6cfc9"} Jul 21 14:31:11.332020 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:11.331984 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" event={"ID":"a5be571b-53ce-44fd-9ae4-aa4cb7e45383","Type":"ContainerStarted","Data":"4c60d39198d8c8d3268d8cc965c0240b79b13b2170797e908ef8892f0ecd0e68"} Jul 21 14:31:11.332301 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:11.332280 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:11.352889 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:11.352802 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" podStartSLOduration=2.352770092 podStartE2EDuration="2.352770092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:31:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:31:11.349846521 +0000 UTC m=+2832.501687631" watchObservedRunningTime="2026-07-21 14:31:11.352770092 +0000 UTC m=+2832.504611206" Jul 21 14:31:11.589781 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:11.589653 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-76b7c599df-jf44z_8ec89ad0-c63e-43cc-bbad-560d9e340689/console/0.log" Jul 21 14:31:12.272639 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:12.272606 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-f2v6c_c7b2425c-081b-45a4-86e2-a0445311d69c/volume-data-source-validator/0.log" Jul 21 14:31:13.421214 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:13.421177 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dtmfw_53a477e4-4535-4679-acbb-197a47d3e114/dns/0.log" Jul 21 14:31:13.443760 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:13.443732 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dtmfw_53a477e4-4535-4679-acbb-197a47d3e114/kube-rbac-proxy/0.log" Jul 21 14:31:13.643347 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:13.643312 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-js4ds_2b19a38a-ffc0-4c33-914c-0eb1b2c67b71/dns-node-resolver/0.log" Jul 21 14:31:14.244971 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:14.244930 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bbnk6_f26b564f-04fd-4e02-ade5-45adcf879547/node-ca/0.log" Jul 21 14:31:15.303813 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:15.303775 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-554689689d996fx_6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5/istio-proxy/0.log" Jul 21 14:31:15.515452 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:15.515418 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74949dcd89-dlm5x_5a886230-716b-4ef6-97ae-0847508d4ab7/istio-proxy/0.log" Jul 21 14:31:15.551525 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:15.551486 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74949dcd89-dlm5x_5a886230-716b-4ef6-97ae-0847508d4ab7/istio-proxy/1.log" Jul 21 14:31:15.613396 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:15.613311 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-f45qg_f668a1ed-b892-4b42-9151-66990c6511e4/payload-processing/0.log" Jul 21 14:31:15.654169 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:15.654141 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5565c7c56b-9stjh_d29c3f6d-9285-48fa-979b-b71b91753fc1/router/0.log" Jul 21 14:31:16.435664 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:16.435636 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p8hzg_a4ff8e96-9fe5-4f87-8f9c-a2d740589863/serve-healthcheck-canary/0.log" Jul 21 14:31:16.930309 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:16.930271 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-65wbr_8d80093e-2150-45d5-b78a-a96386daf7e9/insights-operator/0.log" Jul 21 14:31:16.930626 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:16.930610 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-65wbr_8d80093e-2150-45d5-b78a-a96386daf7e9/insights-operator/1.log" Jul 21 14:31:17.156333 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:17.156304 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v7j4w_c52474d7-8c91-4256-bd47-fd1aef601222/kube-rbac-proxy/0.log" Jul 21 14:31:17.190172 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:17.190076 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v7j4w_c52474d7-8c91-4256-bd47-fd1aef601222/exporter/0.log" Jul 21 14:31:17.224987 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:17.224939 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v7j4w_c52474d7-8c91-4256-bd47-fd1aef601222/extractor/0.log" Jul 21 14:31:17.356047 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:17.356014 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-4mdwf" Jul 21 14:31:20.311117 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:20.310985 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d98d8798d-jmhl7_150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6/manager/0.log" Jul 21 14:31:20.347039 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:20.347005 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d98d8798d-jmhl7_150a2a3c-6487-41eb-8c84-7cc1bd8c4dd6/copy-manifests/0.log" Jul 21 14:31:21.667464 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:21.667335 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-554689689d996fx_6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5/istio-proxy/0.log" Jul 21 14:31:21.816463 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:21.816419 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-686f4b4fc5-mbrz5_991d90c7-3003-4ffe-b861-5a7ad42dbf5d/manager/0.log" Jul 21 14:31:21.842821 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:21.842719 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74949dcd89-dlm5x_5a886230-716b-4ef6-97ae-0847508d4ab7/istio-proxy/0.log" Jul 21 14:31:21.875021 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:21.874896 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-tqdg4_aff57af5-3ad0-419d-a2d9-c8cc29a7e5db/openshift-lws-operator/0.log" Jul 21 14:31:21.883019 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:21.882977 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74949dcd89-dlm5x_5a886230-716b-4ef6-97ae-0847508d4ab7/istio-proxy/1.log" Jul 21 14:31:21.929201 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:21.929133 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-f45qg_f668a1ed-b892-4b42-9151-66990c6511e4/payload-processing/0.log" Jul 21 14:31:21.952360 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:21.952323 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5565c7c56b-9stjh_d29c3f6d-9285-48fa-979b-b71b91753fc1/router/0.log" Jul 21 14:31:23.181398 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:23.181366 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-554689689d996fx_6c857ac6-0465-49f4-ae6f-29f5e7cfb2d5/istio-proxy/0.log" Jul 21 14:31:23.371660 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:23.371622 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74949dcd89-dlm5x_5a886230-716b-4ef6-97ae-0847508d4ab7/istio-proxy/0.log" Jul 21 14:31:23.422515 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:23.422472 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74949dcd89-dlm5x_5a886230-716b-4ef6-97ae-0847508d4ab7/istio-proxy/1.log" Jul 21 14:31:23.478412 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:23.478308 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-f45qg_f668a1ed-b892-4b42-9151-66990c6511e4/payload-processing/0.log" Jul 21 14:31:23.500277 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:23.497811 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5565c7c56b-9stjh_d29c3f6d-9285-48fa-979b-b71b91753fc1/router/0.log" Jul 21 14:31:27.250982 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:27.250946 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-d4t5n_73189f81-d993-4346-9fc5-be10ee914c94/migrator/0.log" Jul 21 14:31:27.279961 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:27.279906 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-d4t5n_73189f81-d993-4346-9fc5-be10ee914c94/graceful-termination/0.log" Jul 21 14:31:27.713484 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:27.713448 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-p56gh_80d7f298-9397-475b-951d-baf26856c339/kube-storage-version-migrator-operator/1.log" Jul 21 14:31:27.714513 ip-10-0-129-212 kubenswrapper[2567]: I0721 14:31:27.714486 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-p56gh_80d7f298-9397-475b-951d-baf26856c339/kube-storage-version-migrator-operator/0.log"