Jul 16 12:20:34.156541 ip-10-0-139-167 systemd[1]: Starting Kubernetes Kubelet... Jul 16 12:20:34.517061 ip-10-0-139-167 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 12:20:34.517061 ip-10-0-139-167 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 12:20:34.517061 ip-10-0-139-167 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 12:20:34.517061 ip-10-0-139-167 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 12:20:34.517061 ip-10-0-139-167 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 12:20:34.517853 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.517769 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 12:20:34.521466 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521451 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:20:34.521466 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521466 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521470 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521473 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521476 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521479 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521481 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521484 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521486 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521489 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521492 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521494 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521497 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521500 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521502 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521505 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521518 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521523 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521525 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521528 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:20:34.521530 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521531 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521533 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521536 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521541 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521545 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521547 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521551 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521553 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521556 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521558 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521561 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521563 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521565 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521568 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521570 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521573 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521575 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521577 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521581 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:20:34.521980 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521583 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521585 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521587 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521590 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521592 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521594 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521597 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521599 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521602 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521604 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521606 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521609 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521611 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521613 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521616 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521619 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521621 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521624 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521626 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521629 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:20:34.522493 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521631 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521634 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521636 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521639 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521641 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521643 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521646 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521648 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521650 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521652 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521655 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521657 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521660 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521663 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521665 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521667 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521670 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521672 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521675 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521678 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:20:34.523081 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521681 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521683 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521685 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521688 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521690 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521692 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521694 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521697 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521700 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521702 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521706 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521710 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521712 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521715 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521718 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521720 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521723 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521725 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521727 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.521730 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:20:34.523592 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522404 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522410 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522413 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522415 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522421 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522431 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522434 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522436 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522439 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522442 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522444 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522446 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522449 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522451 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522454 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522456 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522459 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522461 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522463 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522468 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:20:34.524074 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522471 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522474 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522476 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522479 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522481 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522484 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522486 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522489 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522491 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522493 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522496 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522499 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522501 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522504 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522506 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522509 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522511 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522513 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522523 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522526 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:20:34.524619 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522528 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522531 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522533 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522536 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522539 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522541 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522546 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522550 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522552 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522555 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522558 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522561 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522564 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522566 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522569 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522571 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522574 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522577 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522580 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:20:34.525155 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522582 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522586 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522588 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522591 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522593 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522596 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522598 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522601 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522603 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522606 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522608 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522610 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522614 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522616 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522619 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522622 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522625 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522627 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522629 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522632 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:20:34.525641 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522634 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522637 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522639 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522643 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522646 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522649 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522652 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522655 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522657 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522660 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522663 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522666 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522668 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522671 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522674 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522676 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522679 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522681 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522683 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:20:34.526127 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.522686 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523076 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523081 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523083 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523086 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523090 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523093 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523095 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523098 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523100 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523103 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523105 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523108 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523110 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523113 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523115 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523118 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523121 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523123 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:20:34.526620 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523127 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523130 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523134 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523137 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523140 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523143 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523146 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523149 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523151 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523154 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523156 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523158 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523161 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523163 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523166 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523168 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523171 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523173 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523176 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523180 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:20:34.527088 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523182 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523184 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523187 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523189 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523191 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523194 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523196 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523198 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523201 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523204 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523207 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523209 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523212 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523214 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523216 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523219 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523222 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523224 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523227 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523230 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:20:34.527611 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523232 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523235 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523237 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523239 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523242 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523244 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523246 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523249 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523251 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523254 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523256 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523258 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523270 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523273 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523277 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523280 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523302 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523306 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523310 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:20:34.528086 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523313 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523317 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523320 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523324 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523327 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523329 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523332 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523335 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523338 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523340 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523343 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523346 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523348 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523351 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523353 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523355 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523358 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523362 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523365 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:20:34.528561 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523367 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523370 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.523372 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523438 2577 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523444 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523449 2577 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523454 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523459 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523464 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523468 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523472 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523475 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523478 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523482 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523486 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523489 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523491 2577 flags.go:64] FLAG: --cgroup-root="" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523494 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523497 2577 flags.go:64] FLAG: --client-ca-file="" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523500 2577 flags.go:64] FLAG: --cloud-provider="external" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523502 2577 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523507 2577 flags.go:64] FLAG: --cluster-domain="" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523510 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523512 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 12:20:34.529009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523515 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523518 2577 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523522 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523524 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523527 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523530 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523533 2577 flags.go:64] FLAG: --contention-profiling="false" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523536 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523539 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523542 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523544 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523548 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523551 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523554 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523556 2577 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523559 2577 flags.go:64] FLAG: --enable-server="true" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523564 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523568 2577 flags.go:64] FLAG: --event-burst="100" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523571 2577 flags.go:64] FLAG: --event-qps="50" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523574 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523576 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523579 2577 flags.go:64] FLAG: --eviction-hard="" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523584 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523587 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523589 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 12:20:34.529656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523592 2577 flags.go:64] FLAG: --eviction-soft="" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523595 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523598 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523601 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523604 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523607 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523610 2577 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523612 2577 flags.go:64] FLAG: --feature-gates="" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523616 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523619 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523622 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523625 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523627 2577 flags.go:64] FLAG: --healthz-port="10248" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523630 2577 flags.go:64] FLAG: --help="false" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523633 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523636 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523638 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523641 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523645 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523648 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523651 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523653 2577 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523656 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 12:20:34.530266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523659 2577 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523662 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523665 2577 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523668 2577 flags.go:64] FLAG: --kube-reserved="" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523670 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523673 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523678 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523680 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523683 2577 flags.go:64] FLAG: --lock-file="" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523686 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523689 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523692 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523696 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523699 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523701 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523704 2577 flags.go:64] FLAG: --logging-format="text" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523707 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523710 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523713 2577 flags.go:64] FLAG: --manifest-url="" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523715 2577 flags.go:64] FLAG: --manifest-url-header="" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523719 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523722 2577 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523726 2577 flags.go:64] FLAG: --max-pods="110" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523729 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523731 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 12:20:34.530906 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523734 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523737 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523740 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523742 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523745 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523752 2577 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523755 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523758 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523761 2577 flags.go:64] FLAG: --pod-cidr="" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523767 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523772 2577 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523775 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523778 2577 flags.go:64] FLAG: --pods-per-core="0" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523782 2577 flags.go:64] FLAG: --port="10250" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523785 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523787 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ab9fdc26b181ef76" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523791 2577 flags.go:64] FLAG: --qos-reserved="" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523793 2577 flags.go:64] FLAG: --read-only-port="10255" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523796 2577 flags.go:64] FLAG: --register-node="true" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523799 2577 flags.go:64] FLAG: --register-with-taints="" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523802 2577 flags.go:64] FLAG: --registry-burst="10" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523805 2577 flags.go:64] FLAG: --registry-qps="5" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523808 2577 flags.go:64] FLAG: --reserved-cpus="" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523810 2577 flags.go:64] FLAG: --reserved-memory="" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523813 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 12:20:34.531547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523816 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523819 2577 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523822 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523824 2577 flags.go:64] FLAG: --runonce="false" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523827 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523830 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523833 2577 flags.go:64] FLAG: --seccomp-default="false" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523835 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523838 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523841 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523844 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523846 2577 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523849 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523851 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523854 2577 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523857 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523860 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523863 2577 flags.go:64] FLAG: --system-cgroups="" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523866 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523871 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523874 2577 flags.go:64] FLAG: --tls-cert-file="" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523877 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523881 2577 flags.go:64] FLAG: --tls-min-version="" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523884 2577 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523886 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 12:20:34.532155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523889 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523892 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523895 2577 flags.go:64] FLAG: --v="2" Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523898 2577 flags.go:64] FLAG: --version="false" Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523902 2577 flags.go:64] FLAG: --vmodule="" Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523906 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.523909 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524021 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524025 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524028 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524030 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524033 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524036 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524039 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524041 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524044 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524047 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524049 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524052 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524055 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524057 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:20:34.532823 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524059 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524062 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524064 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524067 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524069 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524072 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524076 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524078 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524081 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524083 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524086 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524088 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524091 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524095 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524098 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524101 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524103 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524106 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524109 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524111 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:20:34.533557 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524114 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524116 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524119 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524121 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524124 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524126 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524129 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524131 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524133 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524136 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524138 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524140 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524142 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524147 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524149 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524151 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524154 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524156 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524163 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:20:34.534227 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524166 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524168 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524170 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524173 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524176 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524178 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524180 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524183 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524187 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524190 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524193 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524208 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524211 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524214 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524216 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524219 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524221 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524224 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524226 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:20:34.534807 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524229 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524231 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524234 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524236 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524238 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524241 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524243 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524247 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524249 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524252 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524254 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524257 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524261 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524264 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524266 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524269 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524271 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524274 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524276 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524279 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:20:34.535303 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524281 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524296 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524299 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524301 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524303 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524306 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.524309 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.524315 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.531230 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.531246 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.531275 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.531281 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531375 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531380 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531383 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:20:34.535830 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531387 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531390 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531392 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531395 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531398 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531401 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531404 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531406 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531408 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531411 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531413 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531416 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531418 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531421 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531423 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531426 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531428 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531432 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531435 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:20:34.536230 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531438 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531441 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531444 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531447 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531449 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531451 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531455 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531465 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531467 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531470 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531472 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531475 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531477 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531480 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531482 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531485 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531487 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531490 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531492 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531494 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:20:34.536706 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531497 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531499 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531502 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531504 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531507 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531509 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531511 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531513 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531516 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531519 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531521 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531523 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531525 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531528 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531530 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531533 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531535 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531538 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531541 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:20:34.537193 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531545 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531554 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531557 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531560 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531563 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531566 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531569 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531572 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531576 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531579 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531582 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531584 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531587 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531590 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531593 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531595 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531598 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531601 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531603 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:20:34.537769 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531606 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531608 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531610 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531613 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531615 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531618 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531620 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531622 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531625 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531627 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531630 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531632 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531635 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531637 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531639 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531647 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531650 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531653 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:20:34.538255 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531660 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.531665 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531808 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531813 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531816 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531818 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531821 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531824 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531826 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531828 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531831 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531833 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531836 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531838 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531840 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531843 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 12:20:34.538744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531845 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531847 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531851 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531855 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531857 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531860 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531862 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531864 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531867 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531869 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531872 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531874 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531876 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531879 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531889 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531893 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531896 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531899 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531902 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 12:20:34.539143 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531905 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531908 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531911 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531914 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531917 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531919 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531921 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531924 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531926 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531929 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531931 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531933 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531936 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531938 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531940 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531943 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531945 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531947 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531950 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 12:20:34.539668 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531952 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531954 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531957 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531959 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531961 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531964 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531966 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531969 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531971 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531979 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531982 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531984 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531987 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531989 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531991 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531994 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531996 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.531998 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532001 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532003 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 12:20:34.540130 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532005 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532008 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532010 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532012 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532015 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532017 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532020 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532022 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532025 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532027 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532029 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532032 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532034 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532037 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532039 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532042 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532044 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532047 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532050 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532052 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 12:20:34.540744 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532054 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 12:20:34.541220 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532057 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 12:20:34.541220 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532064 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 12:20:34.541220 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532067 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 12:20:34.541220 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532069 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 12:20:34.541220 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532072 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 12:20:34.541220 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:34.532074 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 12:20:34.541220 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.532079 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 12:20:34.541220 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.532191 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 12:20:34.541220 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.535571 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 12:20:34.541220 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.537045 2577 server.go:1019] "Starting client certificate rotation" Jul 16 12:20:34.541220 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.537142 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 12:20:34.541220 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.537171 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 12:20:34.541569 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.539523 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 12:20:34.543015 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.542998 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 12:20:34.551001 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.550985 2577 log.go:25] "Validated CRI v1 runtime API" Jul 16 12:20:34.552989 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.552976 2577 log.go:25] "Validated CRI v1 image API" Jul 16 12:20:34.553727 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.553713 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 12:20:34.555847 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.555831 2577 fs.go:135] Filesystem UUIDs: map[4d1a4e03-2f34-479e-a289-b4b3cc01cfb6:/dev/nvme0n1p4 4f9ab06a-529f-46d0-a5a0-0b178da7c7d3:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 16 12:20:34.555893 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.555848 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 12:20:34.561163 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.561062 2577 manager.go:216] Machine: {Timestamp:2026-07-16 12:20:34.560577731 +0000 UTC m=+0.299483714 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099615 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b9ded0be367d87bfe995991b6853a SystemUUID:ec2b9ded-0be3-67d8-7bfe-995991b6853a BootID:bac39b90-24d2-4b0f-8d89-ebb228462fe7 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dc:a4:9f:ec:95 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dc:a4:9f:ec:95 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:54:a4:95:63:47 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 12:20:34.561163 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.561160 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 12:20:34.561267 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.561256 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 12:20:34.563975 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.563951 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 12:20:34.564100 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.563978 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-167.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 12:20:34.564863 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.564848 2577 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 12:20:34.564914 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.564865 2577 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 12:20:34.564914 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.564873 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 12:20:34.565491 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.565480 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 12:20:34.566905 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.566893 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 12:20:34.566989 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.566975 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 12:20:34.570774 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.570761 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 16 12:20:34.570879 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.570870 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 12:20:34.570921 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.570899 2577 kubelet.go:486] "Attempting to sync node with API server" Jul 16 12:20:34.570921 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.570911 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 12:20:34.570983 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.570924 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 12:20:34.570983 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.570933 2577 kubelet.go:398] "Adding apiserver pod source" Jul 16 12:20:34.570983 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.570942 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 12:20:34.571319 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.571276 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 12:20:34.571402 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.571336 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 12:20:34.573751 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.573643 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 12:20:34.575539 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.575523 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 12:20:34.575601 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.575554 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 12:20:34.575710 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.575701 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 12:20:34.575739 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.575715 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 12:20:34.575739 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.575722 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 12:20:34.575739 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.575727 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 12:20:34.575739 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.575735 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 12:20:34.575739 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.575739 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 12:20:34.576219 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.576210 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 12:20:34.576219 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.576218 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 12:20:34.576302 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.576225 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 12:20:34.576302 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.576230 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 12:20:34.576302 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.576246 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 12:20:34.576302 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.576254 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 12:20:34.576302 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.576278 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 12:20:34.576302 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.576296 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 12:20:34.579567 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.579551 2577 server.go:1268] "Started kubelet" Jul 16 12:20:34.579665 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.579612 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 12:20:34.579771 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.579707 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 12:20:34.579843 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.579794 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 12:20:34.580489 ip-10-0-139-167 systemd[1]: Started Kubernetes Kubelet. Jul 16 12:20:34.581441 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.580990 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 12:20:34.581441 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.581119 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-167.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 12:20:34.581441 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.581126 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-167.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 12:20:34.581441 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.581374 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 12:20:34.582356 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.582224 2577 server.go:310] "Adding debug handlers to kubelet server" Jul 16 12:20:34.584347 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.583481 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-167.ec2.internal.18c2c47ae9ca0e3b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-167.ec2.internal,UID:ip-10-0-139-167.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-167.ec2.internal,},FirstTimestamp:2026-07-16 12:20:34.579525179 +0000 UTC m=+0.318431162,LastTimestamp:2026-07-16 12:20:34.579525179 +0000 UTC m=+0.318431162,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-167.ec2.internal,}" Jul 16 12:20:34.585256 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.585240 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 12:20:34.586750 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.586737 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 12:20:34.586816 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.586756 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 12:20:34.586865 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.586814 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:34.587346 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.587333 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 12:20:34.587346 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.587330 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 12:20:34.587483 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.587356 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 12:20:34.587483 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.587450 2577 reconstruct.go:97] "Volume reconstruction finished" Jul 16 12:20:34.587483 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.587460 2577 reconciler.go:29] "Reconciler: start to sync state" Jul 16 12:20:34.587695 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.587677 2577 factory.go:55] Registering systemd factory Jul 16 12:20:34.587695 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.587698 2577 factory.go:223] Registration of the systemd container factory successfully Jul 16 12:20:34.587913 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.587900 2577 factory.go:153] Registering CRI-O factory Jul 16 12:20:34.587913 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.587915 2577 factory.go:223] Registration of the crio container factory successfully Jul 16 12:20:34.587990 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.587985 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 12:20:34.588028 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.588005 2577 factory.go:103] Registering Raw factory Jul 16 12:20:34.588028 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.588016 2577 manager.go:1196] Started watching for new ooms in manager Jul 16 12:20:34.588325 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.588314 2577 manager.go:349] Starting recovery of all containers Jul 16 12:20:34.594341 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.594315 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jvvc7" Jul 16 12:20:34.596783 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.596754 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-167.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 12:20:34.597082 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.597058 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 12:20:34.601014 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.600891 2577 manager.go:354] Recovery completed Jul 16 12:20:34.601924 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.601905 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jvvc7" Jul 16 12:20:34.605170 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.605159 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:20:34.606894 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.606877 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:20:34.606977 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.606910 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:20:34.606977 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.606923 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:20:34.607503 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.607491 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 12:20:34.607503 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.607504 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 12:20:34.607582 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.607519 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 16 12:20:34.607582 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.607539 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:20:34.608172 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.608158 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:20:34.608218 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.608187 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:20:34.608218 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.608202 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:20:34.609092 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.609032 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-167.ec2.internal.18c2c47aeb6baaf2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-167.ec2.internal,UID:ip-10-0-139-167.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-167.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-167.ec2.internal,},FirstTimestamp:2026-07-16 12:20:34.60689381 +0000 UTC m=+0.345799800,LastTimestamp:2026-07-16 12:20:34.60689381 +0000 UTC m=+0.345799800,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-167.ec2.internal,}" Jul 16 12:20:34.609482 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.609472 2577 policy_none.go:49] "None policy: Start" Jul 16 12:20:34.609514 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.609487 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 12:20:34.609514 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.609497 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 12:20:34.611438 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.611428 2577 policy_none.go:47] "Start" Jul 16 12:20:34.646359 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.646345 2577 manager.go:337] "Starting Device Plugin manager" Jul 16 12:20:34.646457 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.646389 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 12:20:34.646457 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.646402 2577 server.go:87] "Starting device plugin registration server" Jul 16 12:20:34.646641 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.646628 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 12:20:34.646693 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.646643 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 12:20:34.646745 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.646726 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 12:20:34.646839 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.646812 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 12:20:34.646839 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.646826 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 12:20:34.647249 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.647231 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 12:20:34.647331 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.647282 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:34.747695 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.747646 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:20:34.749900 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.748946 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:20:34.749900 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.748976 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:20:34.749900 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.748989 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:20:34.749900 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.749016 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.754140 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.754116 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 12:20:34.755489 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.755475 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 12:20:34.755554 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.755509 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 12:20:34.755554 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.755530 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 12:20:34.755641 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.755561 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 12:20:34.757114 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.757094 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.757204 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.757121 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-167.ec2.internal\": node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:34.757797 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.757782 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:20:34.801068 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.801047 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:34.856619 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.856572 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal"] Jul 16 12:20:34.856763 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.856661 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:20:34.857585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.857568 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:20:34.857685 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.857600 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:20:34.857685 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.857616 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:20:34.859912 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.859900 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:20:34.860090 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.860078 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.860131 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.860114 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:20:34.860708 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.860684 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:20:34.860787 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.860724 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:20:34.860787 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.860728 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:20:34.860787 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.860738 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:20:34.860787 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.860753 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:20:34.860787 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.860769 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:20:34.862979 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.862950 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.863047 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.862988 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 12:20:34.863713 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.863697 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientMemory" Jul 16 12:20:34.863781 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.863725 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 12:20:34.863781 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.863739 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeHasSufficientPID" Jul 16 12:20:34.888673 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.888643 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0857f75acb9c2fdf2f3f45bdfb2dc1ec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal\" (UID: \"0857f75acb9c2fdf2f3f45bdfb2dc1ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.888765 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.888676 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0857f75acb9c2fdf2f3f45bdfb2dc1ec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal\" (UID: \"0857f75acb9c2fdf2f3f45bdfb2dc1ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.888765 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.888694 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1bb2e9c0b1e73e2a5eceaddc294c2341-config\") pod \"kube-apiserver-proxy-ip-10-0-139-167.ec2.internal\" (UID: \"1bb2e9c0b1e73e2a5eceaddc294c2341\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.894026 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.894006 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-167.ec2.internal\" not found" node="ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.898408 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.898393 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-167.ec2.internal\" not found" node="ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.901925 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:34.901913 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:34.989427 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.989395 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1bb2e9c0b1e73e2a5eceaddc294c2341-config\") pod \"kube-apiserver-proxy-ip-10-0-139-167.ec2.internal\" (UID: \"1bb2e9c0b1e73e2a5eceaddc294c2341\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.989427 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.989429 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0857f75acb9c2fdf2f3f45bdfb2dc1ec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal\" (UID: \"0857f75acb9c2fdf2f3f45bdfb2dc1ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.989556 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.989450 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0857f75acb9c2fdf2f3f45bdfb2dc1ec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal\" (UID: \"0857f75acb9c2fdf2f3f45bdfb2dc1ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.989556 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.989488 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0857f75acb9c2fdf2f3f45bdfb2dc1ec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal\" (UID: \"0857f75acb9c2fdf2f3f45bdfb2dc1ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.989556 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.989507 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0857f75acb9c2fdf2f3f45bdfb2dc1ec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal\" (UID: \"0857f75acb9c2fdf2f3f45bdfb2dc1ec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jul 16 12:20:34.989556 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:34.989488 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1bb2e9c0b1e73e2a5eceaddc294c2341-config\") pod \"kube-apiserver-proxy-ip-10-0-139-167.ec2.internal\" (UID: \"1bb2e9c0b1e73e2a5eceaddc294c2341\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" Jul 16 12:20:35.002519 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:35.002497 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:35.103371 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:35.103305 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:35.195843 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:35.195807 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jul 16 12:20:35.201819 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:35.201800 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" Jul 16 12:20:35.203927 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:35.203914 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:35.304233 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:35.304193 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:35.404735 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:35.404666 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:35.505257 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:35.505220 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:35.536404 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:35.536379 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 12:20:35.537006 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:35.536516 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 12:20:35.605458 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:35.605423 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:35.606528 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:35.606502 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 12:15:36 +0000 UTC" deadline="2028-03-07 15:29:25.297885728 +0000 UTC" Jul 16 12:20:35.606593 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:35.606528 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14403h8m49.691360927s" Jul 16 12:20:35.690064 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:35.690046 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:20:35.706091 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:35.706060 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:37.683816 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:37.683796 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:37.689217 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:37.689187 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0857f75acb9c2fdf2f3f45bdfb2dc1ec.slice/crio-aefb3c47bc42865b1f65c4da56a3d833e167803b7a30c41cb1965c29d5d0129c WatchSource:0}: Error finding container aefb3c47bc42865b1f65c4da56a3d833e167803b7a30c41cb1965c29d5d0129c: Status 404 returned error can't find the container with id aefb3c47bc42865b1f65c4da56a3d833e167803b7a30c41cb1965c29d5d0129c Jul 16 12:20:37.689705 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:37.689689 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bb2e9c0b1e73e2a5eceaddc294c2341.slice/crio-bce14a896246d8049f5f2e51120e8f613c8d0c4c67cc4417918a2bfe17349270 WatchSource:0}: Error finding container bce14a896246d8049f5f2e51120e8f613c8d0c4c67cc4417918a2bfe17349270: Status 404 returned error can't find the container with id bce14a896246d8049f5f2e51120e8f613c8d0c4c67cc4417918a2bfe17349270 Jul 16 12:20:37.696592 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:37.696578 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 12:20:37.784789 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:37.784751 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:37.885268 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:37.885209 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-167.ec2.internal\" not found" Jul 16 12:20:37.921396 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:37.921072 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:20:37.964195 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:37.964169 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" Jul 16 12:20:37.975208 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:37.975190 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 12:20:37.976973 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:37.976956 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" Jul 16 12:20:37.986052 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:37.986028 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 12:20:38.010338 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.010320 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:20:38.449722 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.449695 2577 apiserver.go:52] "Watching apiserver" Jul 16 12:20:38.456428 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.456406 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 12:20:38.457746 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.457721 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-5xbcz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal","openshift-network-diagnostics/network-check-target-4qrhp","openshift-network-operator/iptables-alerter-9f5jm","openshift-ovn-kubernetes/ovnkube-node-7cwqf","kube-system/global-pull-secret-syncer-wjdcb","kube-system/konnectivity-agent-sw9kn","kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal","openshift-dns/node-resolver-lxhnm","openshift-image-registry/node-ca-mw9kk","openshift-multus/multus-additional-cni-plugins-kxg9t","openshift-multus/multus-jqf5n","openshift-multus/network-metrics-daemon-98bqc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf"] Jul 16 12:20:38.460665 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.460234 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sw9kn" Jul 16 12:20:38.462620 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.462601 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:38.462715 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:38.462668 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:20:38.463660 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.463360 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 12:20:38.463660 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.463386 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 12:20:38.463660 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.463560 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lnbmq\"" Jul 16 12:20:38.464041 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.464001 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 12:20:38.467647 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.464861 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:38.467647 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:38.464922 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:20:38.467647 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.467005 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9f5jm" Jul 16 12:20:38.469673 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.469319 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.469673 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.469424 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j79v6\"" Jul 16 12:20:38.469673 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.469433 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:20:38.469673 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.469504 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 12:20:38.469673 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.469634 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 12:20:38.471548 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.471510 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 12:20:38.471648 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.471601 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 12:20:38.472173 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.471823 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.472173 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.471953 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 12:20:38.472173 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.471954 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 12:20:38.472173 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.471953 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zjbw5\"" Jul 16 12:20:38.472447 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.472412 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 12:20:38.472447 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.472427 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 12:20:38.473899 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.473883 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 12:20:38.474206 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.474125 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 12:20:38.474206 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.474154 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7zwbh\"" Jul 16 12:20:38.475621 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.474131 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.475621 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.474766 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl62k\" (UniqueName: \"kubernetes.io/projected/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-kube-api-access-fl62k\") pod \"network-metrics-daemon-98bqc\" (UID: \"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e\") " pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:38.475621 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.474823 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fjv2\" (UniqueName: \"kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2\") pod \"network-check-target-4qrhp\" (UID: \"ac6ef832-3b6b-4b74-a5bf-3823389a6d1c\") " pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:38.475621 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.474857 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7233e4f2-8303-45b4-8870-3ab003f5e5d8-agent-certs\") pod \"konnectivity-agent-sw9kn\" (UID: \"7233e4f2-8303-45b4-8870-3ab003f5e5d8\") " pod="kube-system/konnectivity-agent-sw9kn" Jul 16 12:20:38.475621 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.474889 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7233e4f2-8303-45b4-8870-3ab003f5e5d8-konnectivity-ca\") pod \"konnectivity-agent-sw9kn\" (UID: \"7233e4f2-8303-45b4-8870-3ab003f5e5d8\") " pod="kube-system/konnectivity-agent-sw9kn" Jul 16 12:20:38.475621 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.474924 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs\") pod \"network-metrics-daemon-98bqc\" (UID: \"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e\") " pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:38.475621 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.475407 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 12:20:38.477698 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.477672 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 12:20:38.477777 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.477697 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 12:20:38.477901 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.477885 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gxd2m\"" Jul 16 12:20:38.478203 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.478121 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.478639 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.478621 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 12:20:38.478873 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.478858 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 12:20:38.479014 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.478893 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 12:20:38.480769 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.480751 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 12:20:38.481163 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.480984 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 12:20:38.481163 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.481000 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4hfhd\"" Jul 16 12:20:38.481508 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.481473 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mw9kk" Jul 16 12:20:38.483591 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.483571 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 12:20:38.483591 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.483587 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mmgnz\"" Jul 16 12:20:38.483931 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.483908 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.484016 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.483936 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 12:20:38.484145 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.484129 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 12:20:38.486016 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.485998 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:20:38.486558 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.486236 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lxhnm" Jul 16 12:20:38.486558 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.486273 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 12:20:38.486558 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.486243 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wzxrr\"" Jul 16 12:20:38.488557 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.488536 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:38.488649 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:38.488599 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:20:38.488706 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.488645 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 12:20:38.488754 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.488539 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 12:20:38.488979 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.488963 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lgbvl\"" Jul 16 12:20:38.500406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.500111 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w8fvm" Jul 16 12:20:38.505975 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.505954 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w8fvm" Jul 16 12:20:38.565137 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.565113 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 12:20:38.575182 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575162 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-systemd-units\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.575271 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575190 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-var-lib-openvswitch\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.575271 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575210 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhz9k\" (UniqueName: \"kubernetes.io/projected/d847d128-22b1-4fcf-a5c2-14d041546993-kube-api-access-jhz9k\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.575271 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575226 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5280509a-3dd9-492d-9eae-02e5c64d56e4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.575271 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575261 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl62k\" (UniqueName: \"kubernetes.io/projected/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-kube-api-access-fl62k\") pod \"network-metrics-daemon-98bqc\" (UID: \"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e\") " pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:38.575434 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575280 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-log-socket\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.575434 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575312 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-device-dir\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.575434 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575335 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-cnibin\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.575434 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575374 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5280509a-3dd9-492d-9eae-02e5c64d56e4-cnibin\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.575434 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575403 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5280509a-3dd9-492d-9eae-02e5c64d56e4-os-release\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.575434 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575425 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5280509a-3dd9-492d-9eae-02e5c64d56e4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.575646 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575443 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-tuned\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.575646 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575472 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a3e09d3-661a-4a7f-8741-f39758ea7303-dbus\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:38.575646 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575510 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-kubelet\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.575646 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575529 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-run-openvswitch\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.575646 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575543 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-etc-kubernetes\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.575646 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575591 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-systemd\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.575646 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575610 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blg6v\" (UniqueName: \"kubernetes.io/projected/926f2417-fb83-42f8-abee-55e41d925420-kube-api-access-blg6v\") pod \"node-resolver-lxhnm\" (UID: \"926f2417-fb83-42f8-abee-55e41d925420\") " pod="openshift-dns/node-resolver-lxhnm" Jul 16 12:20:38.575646 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575632 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a99347fd-ae73-45ca-942e-9d5d91d59c2a-iptables-alerter-script\") pod \"iptables-alerter-9f5jm\" (UID: \"a99347fd-ae73-45ca-942e-9d5d91d59c2a\") " pod="openshift-network-operator/iptables-alerter-9f5jm" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575652 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdgq8\" (UniqueName: \"kubernetes.io/projected/95931072-3e08-422c-aff5-1e519977e11c-kube-api-access-sdgq8\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575668 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f-host\") pod \"node-ca-mw9kk\" (UID: \"d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f\") " pod="openshift-image-registry/node-ca-mw9kk" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575684 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-lib-modules\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575703 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8x5v\" (UniqueName: \"kubernetes.io/projected/f1a80591-5b2d-4604-9121-452bfcf30a3b-kube-api-access-h8x5v\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575720 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-run-ovn\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575733 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-cni-bin\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575747 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-var-lib-cni-bin\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575788 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-run-multus-certs\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575821 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5280509a-3dd9-492d-9eae-02e5c64d56e4-cni-binary-copy\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575849 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-var-lib-kubelet\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575870 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shtqh\" (UniqueName: \"kubernetes.io/projected/a99347fd-ae73-45ca-942e-9d5d91d59c2a-kube-api-access-shtqh\") pod \"iptables-alerter-9f5jm\" (UID: \"a99347fd-ae73-45ca-942e-9d5d91d59c2a\") " pod="openshift-network-operator/iptables-alerter-9f5jm" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575886 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-modprobe-d\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575905 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/926f2417-fb83-42f8-abee-55e41d925420-tmp-dir\") pod \"node-resolver-lxhnm\" (UID: \"926f2417-fb83-42f8-abee-55e41d925420\") " pod="openshift-dns/node-resolver-lxhnm" Jul 16 12:20:38.575919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575920 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-registration-dir\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575941 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-etc-selinux\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575956 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-sys-fs\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575969 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-multus-cni-dir\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.575983 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-run-k8s-cni-cncf-io\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576020 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-hostroot\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576072 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-kubernetes\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576098 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-host\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576122 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/13b833aa-75f5-490c-8df2-b9d2247c61d6-ovnkube-script-lib\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576147 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5280509a-3dd9-492d-9eae-02e5c64d56e4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576171 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-socket-dir\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576199 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs\") pod \"network-metrics-daemon-98bqc\" (UID: \"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e\") " pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576246 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fjv2\" (UniqueName: \"kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2\") pod \"network-check-target-4qrhp\" (UID: \"ac6ef832-3b6b-4b74-a5bf-3823389a6d1c\") " pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576282 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct4n9\" (UniqueName: \"kubernetes.io/projected/d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f-kube-api-access-ct4n9\") pod \"node-ca-mw9kk\" (UID: \"d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f\") " pod="openshift-image-registry/node-ca-mw9kk" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576325 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-sysctl-d\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576347 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-sysctl-conf\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:38.576366 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:20:38.576562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576368 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-run\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:38.576438 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs podName:bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e nodeName:}" failed. No retries permitted until 2026-07-16 12:20:39.076415123 +0000 UTC m=+2.938316348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs") pod "network-metrics-daemon-98bqc" (UID: "bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576478 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-run-netns\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576496 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/13b833aa-75f5-490c-8df2-b9d2247c61d6-ovn-node-metrics-cert\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576533 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a99347fd-ae73-45ca-942e-9d5d91d59c2a-host-slash\") pod \"iptables-alerter-9f5jm\" (UID: \"a99347fd-ae73-45ca-942e-9d5d91d59c2a\") " pod="openshift-network-operator/iptables-alerter-9f5jm" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576554 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-os-release\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576567 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-node-log\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576588 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/13b833aa-75f5-490c-8df2-b9d2247c61d6-env-overrides\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576627 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95931072-3e08-422c-aff5-1e519977e11c-cni-binary-copy\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576650 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-run-netns\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576667 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9l82\" (UniqueName: \"kubernetes.io/projected/5280509a-3dd9-492d-9eae-02e5c64d56e4-kube-api-access-f9l82\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576697 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1a80591-5b2d-4604-9121-452bfcf30a3b-tmp\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-var-lib-cni-multus\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576752 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-var-lib-kubelet\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576794 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5280509a-3dd9-492d-9eae-02e5c64d56e4-system-cni-dir\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576814 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-slash\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.577056 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576830 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-multus-conf-dir\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576859 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7233e4f2-8303-45b4-8870-3ab003f5e5d8-konnectivity-ca\") pod \"konnectivity-agent-sw9kn\" (UID: \"7233e4f2-8303-45b4-8870-3ab003f5e5d8\") " pod="kube-system/konnectivity-agent-sw9kn" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576878 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-run-systemd\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576894 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfgv8\" (UniqueName: \"kubernetes.io/projected/13b833aa-75f5-490c-8df2-b9d2247c61d6-kube-api-access-xfgv8\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576907 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-multus-socket-dir-parent\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576921 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-sys\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576941 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a3e09d3-661a-4a7f-8741-f39758ea7303-kubelet-config\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576959 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576973 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.576988 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-cni-netd\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.577005 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-system-cni-dir\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.577025 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95931072-3e08-422c-aff5-1e519977e11c-multus-daemon-config\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.577054 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.577068 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f-serviceca\") pod \"node-ca-mw9kk\" (UID: \"d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f\") " pod="openshift-image-registry/node-ca-mw9kk" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.577090 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-etc-openvswitch\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.577119 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.577642 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.577142 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/13b833aa-75f5-490c-8df2-b9d2247c61d6-ovnkube-config\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.578087 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.577159 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7233e4f2-8303-45b4-8870-3ab003f5e5d8-agent-certs\") pod \"konnectivity-agent-sw9kn\" (UID: \"7233e4f2-8303-45b4-8870-3ab003f5e5d8\") " pod="kube-system/konnectivity-agent-sw9kn" Jul 16 12:20:38.578087 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.577188 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-sysconfig\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.578087 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.577274 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/926f2417-fb83-42f8-abee-55e41d925420-hosts-file\") pod \"node-resolver-lxhnm\" (UID: \"926f2417-fb83-42f8-abee-55e41d925420\") " pod="openshift-dns/node-resolver-lxhnm" Jul 16 12:20:38.578087 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.577390 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7233e4f2-8303-45b4-8870-3ab003f5e5d8-konnectivity-ca\") pod \"konnectivity-agent-sw9kn\" (UID: \"7233e4f2-8303-45b4-8870-3ab003f5e5d8\") " pod="kube-system/konnectivity-agent-sw9kn" Jul 16 12:20:38.578087 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.577433 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 12:20:38.580366 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.580348 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7233e4f2-8303-45b4-8870-3ab003f5e5d8-agent-certs\") pod \"konnectivity-agent-sw9kn\" (UID: \"7233e4f2-8303-45b4-8870-3ab003f5e5d8\") " pod="kube-system/konnectivity-agent-sw9kn" Jul 16 12:20:38.597023 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:38.596989 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:20:38.597145 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:38.597032 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:20:38.597145 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:38.597053 2577 projected.go:196] Error preparing data for projected volume kube-api-access-6fjv2 for pod openshift-network-diagnostics/network-check-target-4qrhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:20:38.597145 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:38.597134 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2 podName:ac6ef832-3b6b-4b74-a5bf-3823389a6d1c nodeName:}" failed. No retries permitted until 2026-07-16 12:20:39.097109822 +0000 UTC m=+2.959011045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6fjv2" (UniqueName: "kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2") pod "network-check-target-4qrhp" (UID: "ac6ef832-3b6b-4b74-a5bf-3823389a6d1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:20:38.600562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.600529 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl62k\" (UniqueName: \"kubernetes.io/projected/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-kube-api-access-fl62k\") pod \"network-metrics-daemon-98bqc\" (UID: \"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e\") " pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:38.637266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.637193 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" event={"ID":"1bb2e9c0b1e73e2a5eceaddc294c2341","Type":"ContainerStarted","Data":"bce14a896246d8049f5f2e51120e8f613c8d0c4c67cc4417918a2bfe17349270"} Jul 16 12:20:38.638314 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.638276 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" event={"ID":"0857f75acb9c2fdf2f3f45bdfb2dc1ec","Type":"ContainerStarted","Data":"aefb3c47bc42865b1f65c4da56a3d833e167803b7a30c41cb1965c29d5d0129c"} Jul 16 12:20:38.678129 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678094 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-kubernetes\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.678315 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678144 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-host\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.678315 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678175 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/13b833aa-75f5-490c-8df2-b9d2247c61d6-ovnkube-script-lib\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.678315 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678232 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-kubernetes\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.678315 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678248 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-host\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.678315 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678284 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5280509a-3dd9-492d-9eae-02e5c64d56e4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.678583 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678332 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-socket-dir\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.678583 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678378 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct4n9\" (UniqueName: \"kubernetes.io/projected/d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f-kube-api-access-ct4n9\") pod \"node-ca-mw9kk\" (UID: \"d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f\") " pod="openshift-image-registry/node-ca-mw9kk" Jul 16 12:20:38.678583 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678403 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-sysctl-d\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.678583 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678428 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-sysctl-conf\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.678583 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678449 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-run\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.678583 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678472 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-run-netns\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.678583 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678495 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/13b833aa-75f5-490c-8df2-b9d2247c61d6-ovn-node-metrics-cert\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.678583 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678511 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-socket-dir\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.678583 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678520 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a99347fd-ae73-45ca-942e-9d5d91d59c2a-host-slash\") pod \"iptables-alerter-9f5jm\" (UID: \"a99347fd-ae73-45ca-942e-9d5d91d59c2a\") " pod="openshift-network-operator/iptables-alerter-9f5jm" Jul 16 12:20:38.678583 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678560 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a99347fd-ae73-45ca-942e-9d5d91d59c2a-host-slash\") pod \"iptables-alerter-9f5jm\" (UID: \"a99347fd-ae73-45ca-942e-9d5d91d59c2a\") " pod="openshift-network-operator/iptables-alerter-9f5jm" Jul 16 12:20:38.678583 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678563 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-os-release\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678611 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-node-log\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678615 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-run\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678626 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-sysctl-conf\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678660 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-run-netns\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678635 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/13b833aa-75f5-490c-8df2-b9d2247c61d6-env-overrides\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678676 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-node-log\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678626 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-sysctl-d\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678698 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95931072-3e08-422c-aff5-1e519977e11c-cni-binary-copy\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678716 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-os-release\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678735 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-run-netns\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678765 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9l82\" (UniqueName: \"kubernetes.io/projected/5280509a-3dd9-492d-9eae-02e5c64d56e4-kube-api-access-f9l82\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1a80591-5b2d-4604-9121-452bfcf30a3b-tmp\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678821 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-var-lib-cni-multus\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678846 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-var-lib-kubelet\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678871 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5280509a-3dd9-492d-9eae-02e5c64d56e4-system-cni-dir\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678895 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-slash\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678895 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5280509a-3dd9-492d-9eae-02e5c64d56e4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.679085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678917 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-multus-conf-dir\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678947 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-run-systemd\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678979 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfgv8\" (UniqueName: \"kubernetes.io/projected/13b833aa-75f5-490c-8df2-b9d2247c61d6-kube-api-access-xfgv8\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678998 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-multus-socket-dir-parent\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679019 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-sys\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679070 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a3e09d3-661a-4a7f-8741-f39758ea7303-kubelet-config\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679093 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679112 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/13b833aa-75f5-490c-8df2-b9d2247c61d6-env-overrides\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679145 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-multus-conf-dir\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679162 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-run-systemd\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679179 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679116 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679200 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-var-lib-cni-multus\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679227 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-cni-netd\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679238 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-slash\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679249 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95931072-3e08-422c-aff5-1e519977e11c-cni-binary-copy\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.678845 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-run-netns\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679253 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-system-cni-dir\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.679963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679199 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-var-lib-kubelet\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679306 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a3e09d3-661a-4a7f-8741-f39758ea7303-kubelet-config\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679321 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-cni-netd\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679254 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-sys\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679354 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/13b833aa-75f5-490c-8df2-b9d2247c61d6-ovnkube-script-lib\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679357 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5280509a-3dd9-492d-9eae-02e5c64d56e4-system-cni-dir\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679378 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-system-cni-dir\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679466 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-multus-socket-dir-parent\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679468 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95931072-3e08-422c-aff5-1e519977e11c-multus-daemon-config\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:38.679504 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679545 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:38.679563 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret podName:4a3e09d3-661a-4a7f-8741-f39758ea7303 nodeName:}" failed. No retries permitted until 2026-07-16 12:20:39.179544672 +0000 UTC m=+3.041445895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret") pod "global-pull-secret-syncer-wjdcb" (UID: "4a3e09d3-661a-4a7f-8741-f39758ea7303") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679598 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679635 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f-serviceca\") pod \"node-ca-mw9kk\" (UID: \"d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f\") " pod="openshift-image-registry/node-ca-mw9kk" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679662 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-etc-openvswitch\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679689 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679716 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/13b833aa-75f5-490c-8df2-b9d2247c61d6-ovnkube-config\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.680814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679749 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-etc-openvswitch\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679753 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679829 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-sysconfig\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679874 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/926f2417-fb83-42f8-abee-55e41d925420-hosts-file\") pod \"node-resolver-lxhnm\" (UID: \"926f2417-fb83-42f8-abee-55e41d925420\") " pod="openshift-dns/node-resolver-lxhnm" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679907 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-sysconfig\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679954 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-systemd-units\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.679984 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-systemd-units\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680033 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-var-lib-openvswitch\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680066 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-var-lib-openvswitch\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680111 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhz9k\" (UniqueName: \"kubernetes.io/projected/d847d128-22b1-4fcf-a5c2-14d041546993-kube-api-access-jhz9k\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680135 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95931072-3e08-422c-aff5-1e519977e11c-multus-daemon-config\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680140 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f-serviceca\") pod \"node-ca-mw9kk\" (UID: \"d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f\") " pod="openshift-image-registry/node-ca-mw9kk" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680209 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/13b833aa-75f5-490c-8df2-b9d2247c61d6-ovnkube-config\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680139 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5280509a-3dd9-492d-9eae-02e5c64d56e4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680262 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-log-socket\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680319 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-device-dir\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680331 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-log-socket\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.681585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680346 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-cnibin\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680335 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/926f2417-fb83-42f8-abee-55e41d925420-hosts-file\") pod \"node-resolver-lxhnm\" (UID: \"926f2417-fb83-42f8-abee-55e41d925420\") " pod="openshift-dns/node-resolver-lxhnm" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680386 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-device-dir\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680372 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5280509a-3dd9-492d-9eae-02e5c64d56e4-cnibin\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680417 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-cnibin\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680429 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5280509a-3dd9-492d-9eae-02e5c64d56e4-os-release\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680439 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5280509a-3dd9-492d-9eae-02e5c64d56e4-cnibin\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680460 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5280509a-3dd9-492d-9eae-02e5c64d56e4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680479 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5280509a-3dd9-492d-9eae-02e5c64d56e4-os-release\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680485 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-tuned\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680507 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a3e09d3-661a-4a7f-8741-f39758ea7303-dbus\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680530 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-kubelet\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680558 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-run-openvswitch\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680568 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5280509a-3dd9-492d-9eae-02e5c64d56e4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680582 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-etc-kubernetes\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680607 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-systemd\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680614 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-kubelet\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.682107 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680631 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blg6v\" (UniqueName: \"kubernetes.io/projected/926f2417-fb83-42f8-abee-55e41d925420-kube-api-access-blg6v\") pod \"node-resolver-lxhnm\" (UID: \"926f2417-fb83-42f8-abee-55e41d925420\") " pod="openshift-dns/node-resolver-lxhnm" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680638 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5280509a-3dd9-492d-9eae-02e5c64d56e4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680644 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a3e09d3-661a-4a7f-8741-f39758ea7303-dbus\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680655 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a99347fd-ae73-45ca-942e-9d5d91d59c2a-iptables-alerter-script\") pod \"iptables-alerter-9f5jm\" (UID: \"a99347fd-ae73-45ca-942e-9d5d91d59c2a\") " pod="openshift-network-operator/iptables-alerter-9f5jm" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680658 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-etc-kubernetes\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680689 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-run-openvswitch\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680700 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-systemd\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680750 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdgq8\" (UniqueName: \"kubernetes.io/projected/95931072-3e08-422c-aff5-1e519977e11c-kube-api-access-sdgq8\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680775 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f-host\") pod \"node-ca-mw9kk\" (UID: \"d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f\") " pod="openshift-image-registry/node-ca-mw9kk" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680796 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-lib-modules\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680819 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8x5v\" (UniqueName: \"kubernetes.io/projected/f1a80591-5b2d-4604-9121-452bfcf30a3b-kube-api-access-h8x5v\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680841 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-run-ovn\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680873 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-cni-bin\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680886 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f-host\") pod \"node-ca-mw9kk\" (UID: \"d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f\") " pod="openshift-image-registry/node-ca-mw9kk" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680896 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-var-lib-cni-bin\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680926 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-run-ovn\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680926 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-run-multus-certs\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680964 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/13b833aa-75f5-490c-8df2-b9d2247c61d6-host-cni-bin\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.682719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680925 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-lib-modules\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.680985 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5280509a-3dd9-492d-9eae-02e5c64d56e4-cni-binary-copy\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681013 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-var-lib-kubelet\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shtqh\" (UniqueName: \"kubernetes.io/projected/a99347fd-ae73-45ca-942e-9d5d91d59c2a-kube-api-access-shtqh\") pod \"iptables-alerter-9f5jm\" (UID: \"a99347fd-ae73-45ca-942e-9d5d91d59c2a\") " pod="openshift-network-operator/iptables-alerter-9f5jm" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681070 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-modprobe-d\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681095 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/926f2417-fb83-42f8-abee-55e41d925420-tmp-dir\") pod \"node-resolver-lxhnm\" (UID: \"926f2417-fb83-42f8-abee-55e41d925420\") " pod="openshift-dns/node-resolver-lxhnm" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-registration-dir\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681141 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-etc-selinux\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681148 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-run-multus-certs\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681164 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-sys-fs\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681175 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a99347fd-ae73-45ca-942e-9d5d91d59c2a-iptables-alerter-script\") pod \"iptables-alerter-9f5jm\" (UID: \"a99347fd-ae73-45ca-942e-9d5d91d59c2a\") " pod="openshift-network-operator/iptables-alerter-9f5jm" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681203 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-var-lib-cni-bin\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681220 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-sys-fs\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681233 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-multus-cni-dir\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681236 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-var-lib-kubelet\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681260 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-run-k8s-cni-cncf-io\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681308 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-hostroot\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.683361 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681314 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-registration-dir\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.683950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681329 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-modprobe-d\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.683950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681359 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-multus-cni-dir\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.683950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681445 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-host-run-k8s-cni-cncf-io\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.683950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681467 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95931072-3e08-422c-aff5-1e519977e11c-hostroot\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.683950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681501 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d847d128-22b1-4fcf-a5c2-14d041546993-etc-selinux\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.683950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681534 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5280509a-3dd9-492d-9eae-02e5c64d56e4-cni-binary-copy\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.683950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681632 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/926f2417-fb83-42f8-abee-55e41d925420-tmp-dir\") pod \"node-resolver-lxhnm\" (UID: \"926f2417-fb83-42f8-abee-55e41d925420\") " pod="openshift-dns/node-resolver-lxhnm" Jul 16 12:20:38.683950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.681759 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/13b833aa-75f5-490c-8df2-b9d2247c61d6-ovn-node-metrics-cert\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.683950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.683201 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f1a80591-5b2d-4604-9121-452bfcf30a3b-etc-tuned\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.683950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.683632 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1a80591-5b2d-4604-9121-452bfcf30a3b-tmp\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.698406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.698365 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blg6v\" (UniqueName: \"kubernetes.io/projected/926f2417-fb83-42f8-abee-55e41d925420-kube-api-access-blg6v\") pod \"node-resolver-lxhnm\" (UID: \"926f2417-fb83-42f8-abee-55e41d925420\") " pod="openshift-dns/node-resolver-lxhnm" Jul 16 12:20:38.708452 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.708363 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdgq8\" (UniqueName: \"kubernetes.io/projected/95931072-3e08-422c-aff5-1e519977e11c-kube-api-access-sdgq8\") pod \"multus-jqf5n\" (UID: \"95931072-3e08-422c-aff5-1e519977e11c\") " pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.708452 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.708437 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shtqh\" (UniqueName: \"kubernetes.io/projected/a99347fd-ae73-45ca-942e-9d5d91d59c2a-kube-api-access-shtqh\") pod \"iptables-alerter-9f5jm\" (UID: \"a99347fd-ae73-45ca-942e-9d5d91d59c2a\") " pod="openshift-network-operator/iptables-alerter-9f5jm" Jul 16 12:20:38.708644 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.708591 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhz9k\" (UniqueName: \"kubernetes.io/projected/d847d128-22b1-4fcf-a5c2-14d041546993-kube-api-access-jhz9k\") pod \"aws-ebs-csi-driver-node-dmgzf\" (UID: \"d847d128-22b1-4fcf-a5c2-14d041546993\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.708954 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.708919 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct4n9\" (UniqueName: \"kubernetes.io/projected/d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f-kube-api-access-ct4n9\") pod \"node-ca-mw9kk\" (UID: \"d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f\") " pod="openshift-image-registry/node-ca-mw9kk" Jul 16 12:20:38.709104 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.709084 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfgv8\" (UniqueName: \"kubernetes.io/projected/13b833aa-75f5-490c-8df2-b9d2247c61d6-kube-api-access-xfgv8\") pod \"ovnkube-node-7cwqf\" (UID: \"13b833aa-75f5-490c-8df2-b9d2247c61d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.710449 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.710431 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9l82\" (UniqueName: \"kubernetes.io/projected/5280509a-3dd9-492d-9eae-02e5c64d56e4-kube-api-access-f9l82\") pod \"multus-additional-cni-plugins-kxg9t\" (UID: \"5280509a-3dd9-492d-9eae-02e5c64d56e4\") " pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.711566 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.711546 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8x5v\" (UniqueName: \"kubernetes.io/projected/f1a80591-5b2d-4604-9121-452bfcf30a3b-kube-api-access-h8x5v\") pod \"tuned-5xbcz\" (UID: \"f1a80591-5b2d-4604-9121-452bfcf30a3b\") " pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.771918 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.771888 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sw9kn" Jul 16 12:20:38.780569 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.780545 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9f5jm" Jul 16 12:20:38.790911 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.790891 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:20:38.796569 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.796551 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" Jul 16 12:20:38.805777 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.805757 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jqf5n" Jul 16 12:20:38.812280 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.812249 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kxg9t" Jul 16 12:20:38.820893 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.820873 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mw9kk" Jul 16 12:20:38.825566 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.825550 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" Jul 16 12:20:38.838079 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.838059 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lxhnm" Jul 16 12:20:38.931181 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:38.931151 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 12:20:39.084569 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.084488 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs\") pod \"network-metrics-daemon-98bqc\" (UID: \"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e\") " pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:39.084710 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:39.084684 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:20:39.084766 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:39.084746 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs podName:bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e nodeName:}" failed. No retries permitted until 2026-07-16 12:20:40.084731735 +0000 UTC m=+3.946632957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs") pod "network-metrics-daemon-98bqc" (UID: "bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:20:39.185099 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.185048 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:39.185250 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.185144 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fjv2\" (UniqueName: \"kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2\") pod \"network-check-target-4qrhp\" (UID: \"ac6ef832-3b6b-4b74-a5bf-3823389a6d1c\") " pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:39.185340 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:39.185308 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:20:39.185340 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:39.185327 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:20:39.185340 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:39.185340 2577 projected.go:196] Error preparing data for projected volume kube-api-access-6fjv2 for pod openshift-network-diagnostics/network-check-target-4qrhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:20:39.185491 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:39.185423 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2 podName:ac6ef832-3b6b-4b74-a5bf-3823389a6d1c nodeName:}" failed. No retries permitted until 2026-07-16 12:20:40.18540829 +0000 UTC m=+4.047309513 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6fjv2" (UniqueName: "kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2") pod "network-check-target-4qrhp" (UID: "ac6ef832-3b6b-4b74-a5bf-3823389a6d1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:20:39.185491 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:39.185447 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:20:39.185595 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:39.185537 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret podName:4a3e09d3-661a-4a7f-8741-f39758ea7303 nodeName:}" failed. No retries permitted until 2026-07-16 12:20:40.185517712 +0000 UTC m=+4.047418936 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret") pod "global-pull-secret-syncer-wjdcb" (UID: "4a3e09d3-661a-4a7f-8741-f39758ea7303") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:20:39.289908 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:39.289690 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95931072_3e08_422c_aff5_1e519977e11c.slice/crio-ae7b0cc7eacd24ce0c58f774ecb7a0d8fc98239a6e79ea33d511affc9d3779e0 WatchSource:0}: Error finding container ae7b0cc7eacd24ce0c58f774ecb7a0d8fc98239a6e79ea33d511affc9d3779e0: Status 404 returned error can't find the container with id ae7b0cc7eacd24ce0c58f774ecb7a0d8fc98239a6e79ea33d511affc9d3779e0 Jul 16 12:20:39.312097 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:39.312074 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd847d128_22b1_4fcf_a5c2_14d041546993.slice/crio-b97cfd58d19a534d4d2a2e314563825ac96b3d11dfb90d3cb43a208925b5e464 WatchSource:0}: Error finding container b97cfd58d19a534d4d2a2e314563825ac96b3d11dfb90d3cb43a208925b5e464: Status 404 returned error can't find the container with id b97cfd58d19a534d4d2a2e314563825ac96b3d11dfb90d3cb43a208925b5e464 Jul 16 12:20:39.312586 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:39.312559 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7233e4f2_8303_45b4_8870_3ab003f5e5d8.slice/crio-c6cb8e00865898ef0ff1d2372e27fdf70b541835220db801ec43c9383915051f WatchSource:0}: Error finding container c6cb8e00865898ef0ff1d2372e27fdf70b541835220db801ec43c9383915051f: Status 404 returned error can't find the container with id c6cb8e00865898ef0ff1d2372e27fdf70b541835220db801ec43c9383915051f Jul 16 12:20:39.313342 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:39.313317 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod926f2417_fb83_42f8_abee_55e41d925420.slice/crio-6866fb47de91c1844668cf6f9dfd52cbab27a034bcc61cbf3c3085cd9b03ff9c WatchSource:0}: Error finding container 6866fb47de91c1844668cf6f9dfd52cbab27a034bcc61cbf3c3085cd9b03ff9c: Status 404 returned error can't find the container with id 6866fb47de91c1844668cf6f9dfd52cbab27a034bcc61cbf3c3085cd9b03ff9c Jul 16 12:20:39.318854 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:20:39.318833 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda99347fd_ae73_45ca_942e_9d5d91d59c2a.slice/crio-281d53304c977c45bcedf68f9e7c638a1bf9467fc99062bbe5610c1fa832cfc9 WatchSource:0}: Error finding container 281d53304c977c45bcedf68f9e7c638a1bf9467fc99062bbe5610c1fa832cfc9: Status 404 returned error can't find the container with id 281d53304c977c45bcedf68f9e7c638a1bf9467fc99062bbe5610c1fa832cfc9 Jul 16 12:20:39.509450 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.509368 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 12:15:38 +0000 UTC" deadline="2028-01-28 00:13:54.305063337 +0000 UTC" Jul 16 12:20:39.509450 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.509443 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13451h53m14.795624615s" Jul 16 12:20:39.633263 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.633191 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:39.633438 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:39.633322 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:20:39.642015 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.641984 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sw9kn" event={"ID":"7233e4f2-8303-45b4-8870-3ab003f5e5d8","Type":"ContainerStarted","Data":"c6cb8e00865898ef0ff1d2372e27fdf70b541835220db801ec43c9383915051f"} Jul 16 12:20:39.643251 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.643194 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jqf5n" event={"ID":"95931072-3e08-422c-aff5-1e519977e11c","Type":"ContainerStarted","Data":"ae7b0cc7eacd24ce0c58f774ecb7a0d8fc98239a6e79ea33d511affc9d3779e0"} Jul 16 12:20:39.645359 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.645332 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" event={"ID":"1bb2e9c0b1e73e2a5eceaddc294c2341","Type":"ContainerStarted","Data":"90d0a1dd032206660491cf50c630f07a3209c1fbfbf23107afe6e9bac88d5377"} Jul 16 12:20:39.646500 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.646479 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mw9kk" event={"ID":"d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f","Type":"ContainerStarted","Data":"544ea5f13d8056ece7cad2d27e0ecf0f59c5119db01999cb4f4230836cdbd9f9"} Jul 16 12:20:39.647743 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.647719 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" event={"ID":"13b833aa-75f5-490c-8df2-b9d2247c61d6","Type":"ContainerStarted","Data":"29ec8b5b04c16cb5aa8ca257283db07d65deacd4209a7acef05fd0bcd5e6155a"} Jul 16 12:20:39.649062 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.649041 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9f5jm" event={"ID":"a99347fd-ae73-45ca-942e-9d5d91d59c2a","Type":"ContainerStarted","Data":"281d53304c977c45bcedf68f9e7c638a1bf9467fc99062bbe5610c1fa832cfc9"} Jul 16 12:20:39.650199 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.650180 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lxhnm" event={"ID":"926f2417-fb83-42f8-abee-55e41d925420","Type":"ContainerStarted","Data":"6866fb47de91c1844668cf6f9dfd52cbab27a034bcc61cbf3c3085cd9b03ff9c"} Jul 16 12:20:39.651503 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.651478 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" event={"ID":"d847d128-22b1-4fcf-a5c2-14d041546993","Type":"ContainerStarted","Data":"b97cfd58d19a534d4d2a2e314563825ac96b3d11dfb90d3cb43a208925b5e464"} Jul 16 12:20:39.652711 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.652691 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kxg9t" event={"ID":"5280509a-3dd9-492d-9eae-02e5c64d56e4","Type":"ContainerStarted","Data":"e824384d510ede3de26e7f87e54cfd1dd0f4819e530d217c14406bc5cb33ae78"} Jul 16 12:20:39.653851 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.653829 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" event={"ID":"f1a80591-5b2d-4604-9121-452bfcf30a3b","Type":"ContainerStarted","Data":"3ebaa7a0598dbbd6e84408a571e4aef8a15f8cd59b7cb225c51fbb913ce9791c"} Jul 16 12:20:39.660598 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:39.660554 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-167.ec2.internal" podStartSLOduration=2.660542043 podStartE2EDuration="2.660542043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:20:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:20:39.660310872 +0000 UTC m=+3.522212294" watchObservedRunningTime="2026-07-16 12:20:39.660542043 +0000 UTC m=+3.522443288" Jul 16 12:20:40.092445 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:40.092414 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs\") pod \"network-metrics-daemon-98bqc\" (UID: \"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e\") " pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:40.092593 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:40.092577 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:20:40.092656 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:40.092638 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs podName:bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e nodeName:}" failed. No retries permitted until 2026-07-16 12:20:42.092620442 +0000 UTC m=+5.954521667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs") pod "network-metrics-daemon-98bqc" (UID: "bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:20:40.192858 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:40.192776 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fjv2\" (UniqueName: \"kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2\") pod \"network-check-target-4qrhp\" (UID: \"ac6ef832-3b6b-4b74-a5bf-3823389a6d1c\") " pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:40.192858 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:40.192845 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:40.193059 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:40.192965 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:20:40.193059 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:40.193030 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret podName:4a3e09d3-661a-4a7f-8741-f39758ea7303 nodeName:}" failed. No retries permitted until 2026-07-16 12:20:42.193011271 +0000 UTC m=+6.054912495 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret") pod "global-pull-secret-syncer-wjdcb" (UID: "4a3e09d3-661a-4a7f-8741-f39758ea7303") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:20:40.193715 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:40.193581 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:20:40.193715 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:40.193605 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:20:40.193715 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:40.193617 2577 projected.go:196] Error preparing data for projected volume kube-api-access-6fjv2 for pod openshift-network-diagnostics/network-check-target-4qrhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:20:40.193715 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:40.193675 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2 podName:ac6ef832-3b6b-4b74-a5bf-3823389a6d1c nodeName:}" failed. No retries permitted until 2026-07-16 12:20:42.193660165 +0000 UTC m=+6.055561406 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6fjv2" (UniqueName: "kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2") pod "network-check-target-4qrhp" (UID: "ac6ef832-3b6b-4b74-a5bf-3823389a6d1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:20:40.509830 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:40.509712 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 12:15:38 +0000 UTC" deadline="2027-12-13 01:06:05.91571921 +0000 UTC" Jul 16 12:20:40.509830 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:40.509749 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12348h45m25.405974156s" Jul 16 12:20:40.635927 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:40.633596 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:40.635927 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:40.633742 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:20:40.635927 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:40.634130 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:40.635927 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:40.634220 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:20:40.661562 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:40.661308 2577 generic.go:356] "Generic (PLEG): container finished" podID="0857f75acb9c2fdf2f3f45bdfb2dc1ec" containerID="cac1ec3e7e8099d1f648459d7ee74475661885c4dcd8dd90ccbf05e23c4e7930" exitCode=0 Jul 16 12:20:40.668228 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:40.664421 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" event={"ID":"0857f75acb9c2fdf2f3f45bdfb2dc1ec","Type":"ContainerDied","Data":"cac1ec3e7e8099d1f648459d7ee74475661885c4dcd8dd90ccbf05e23c4e7930"} Jul 16 12:20:41.633455 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:41.632956 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:41.633455 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:41.633085 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:20:41.683574 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:41.683528 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" event={"ID":"0857f75acb9c2fdf2f3f45bdfb2dc1ec","Type":"ContainerStarted","Data":"ff6ce286405b0c5433b2ee9ebbcd44807b9ac745c4392db9c683d765823eaf51"} Jul 16 12:20:42.109205 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:42.109165 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs\") pod \"network-metrics-daemon-98bqc\" (UID: \"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e\") " pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:42.109397 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:42.109347 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:20:42.109474 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:42.109410 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs podName:bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e nodeName:}" failed. No retries permitted until 2026-07-16 12:20:46.109391127 +0000 UTC m=+9.971292350 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs") pod "network-metrics-daemon-98bqc" (UID: "bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:20:42.209724 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:42.209685 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fjv2\" (UniqueName: \"kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2\") pod \"network-check-target-4qrhp\" (UID: \"ac6ef832-3b6b-4b74-a5bf-3823389a6d1c\") " pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:42.209907 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:42.209768 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:42.209973 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:42.209903 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:20:42.209973 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:42.209965 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret podName:4a3e09d3-661a-4a7f-8741-f39758ea7303 nodeName:}" failed. No retries permitted until 2026-07-16 12:20:46.209946372 +0000 UTC m=+10.071847598 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret") pod "global-pull-secret-syncer-wjdcb" (UID: "4a3e09d3-661a-4a7f-8741-f39758ea7303") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:20:42.210430 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:42.210379 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:20:42.210430 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:42.210402 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:20:42.210430 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:42.210415 2577 projected.go:196] Error preparing data for projected volume kube-api-access-6fjv2 for pod openshift-network-diagnostics/network-check-target-4qrhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:20:42.210618 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:42.210455 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2 podName:ac6ef832-3b6b-4b74-a5bf-3823389a6d1c nodeName:}" failed. No retries permitted until 2026-07-16 12:20:46.210442912 +0000 UTC m=+10.072344135 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6fjv2" (UniqueName: "kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2") pod "network-check-target-4qrhp" (UID: "ac6ef832-3b6b-4b74-a5bf-3823389a6d1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:20:42.633505 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:42.633436 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:42.633943 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:42.633579 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:20:42.633943 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:42.633617 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:42.633943 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:42.633686 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:20:43.633013 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:43.632977 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:43.633197 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:43.633119 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:20:44.633403 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:44.633373 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:44.633823 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:44.633506 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:20:44.633823 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:44.633697 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:44.633823 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:44.633776 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:20:45.633043 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:45.633008 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:45.633277 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:45.633139 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:20:46.139887 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:46.139851 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs\") pod \"network-metrics-daemon-98bqc\" (UID: \"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e\") " pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:46.140374 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:46.140002 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:20:46.140374 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:46.140063 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs podName:bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e nodeName:}" failed. No retries permitted until 2026-07-16 12:20:54.140044659 +0000 UTC m=+18.001945885 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs") pod "network-metrics-daemon-98bqc" (UID: "bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:20:46.240261 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:46.240205 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:46.240488 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:46.240359 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fjv2\" (UniqueName: \"kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2\") pod \"network-check-target-4qrhp\" (UID: \"ac6ef832-3b6b-4b74-a5bf-3823389a6d1c\") " pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:46.240488 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:46.240388 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:20:46.240488 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:46.240454 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret podName:4a3e09d3-661a-4a7f-8741-f39758ea7303 nodeName:}" failed. No retries permitted until 2026-07-16 12:20:54.240439799 +0000 UTC m=+18.102341023 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret") pod "global-pull-secret-syncer-wjdcb" (UID: "4a3e09d3-661a-4a7f-8741-f39758ea7303") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:20:46.240488 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:46.240489 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:20:46.240730 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:46.240508 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:20:46.240730 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:46.240521 2577 projected.go:196] Error preparing data for projected volume kube-api-access-6fjv2 for pod openshift-network-diagnostics/network-check-target-4qrhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:20:46.240730 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:46.240574 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2 podName:ac6ef832-3b6b-4b74-a5bf-3823389a6d1c nodeName:}" failed. No retries permitted until 2026-07-16 12:20:54.240558996 +0000 UTC m=+18.102460222 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6fjv2" (UniqueName: "kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2") pod "network-check-target-4qrhp" (UID: "ac6ef832-3b6b-4b74-a5bf-3823389a6d1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:20:46.634323 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:46.633722 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:46.634323 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:46.633825 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:20:46.634323 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:46.634197 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:46.634323 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:46.634307 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:20:47.633854 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:47.633671 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:47.633854 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:47.633808 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:20:48.633001 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:48.632970 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:48.633191 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:48.633098 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:20:48.633191 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:48.633176 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:48.633321 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:48.633284 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:20:49.632821 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:49.632783 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:49.633205 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:49.632903 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:20:50.636241 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:50.636195 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:50.636659 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:50.636220 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:50.636659 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:50.636339 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:20:50.636659 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:50.636407 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:20:51.633464 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:51.633427 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:51.633609 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:51.633550 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:20:52.633948 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:52.633501 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:52.633948 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:52.633623 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:20:52.633948 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:52.633684 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:52.633948 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:52.633794 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:20:53.632763 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:53.632728 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:53.632983 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:53.632847 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:20:54.199460 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:54.199415 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs\") pod \"network-metrics-daemon-98bqc\" (UID: \"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e\") " pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:54.199945 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:54.199576 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:20:54.199945 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:54.199644 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs podName:bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e nodeName:}" failed. No retries permitted until 2026-07-16 12:21:10.199628214 +0000 UTC m=+34.061529438 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs") pod "network-metrics-daemon-98bqc" (UID: "bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:20:54.300627 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:54.300596 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:54.300798 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:54.300653 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fjv2\" (UniqueName: \"kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2\") pod \"network-check-target-4qrhp\" (UID: \"ac6ef832-3b6b-4b74-a5bf-3823389a6d1c\") " pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:54.300798 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:54.300766 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:20:54.300798 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:54.300767 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:20:54.300798 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:54.300779 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:20:54.300798 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:54.300794 2577 projected.go:196] Error preparing data for projected volume kube-api-access-6fjv2 for pod openshift-network-diagnostics/network-check-target-4qrhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:20:54.301041 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:54.300837 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2 podName:ac6ef832-3b6b-4b74-a5bf-3823389a6d1c nodeName:}" failed. No retries permitted until 2026-07-16 12:21:10.300824866 +0000 UTC m=+34.162726088 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6fjv2" (UniqueName: "kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2") pod "network-check-target-4qrhp" (UID: "ac6ef832-3b6b-4b74-a5bf-3823389a6d1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:20:54.301041 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:54.300850 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret podName:4a3e09d3-661a-4a7f-8741-f39758ea7303 nodeName:}" failed. No retries permitted until 2026-07-16 12:21:10.300844163 +0000 UTC m=+34.162745387 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret") pod "global-pull-secret-syncer-wjdcb" (UID: "4a3e09d3-661a-4a7f-8741-f39758ea7303") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:20:54.633230 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:54.633149 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:54.633230 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:54.633192 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:54.633465 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:54.633285 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:20:54.633465 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:54.633384 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:20:55.633089 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:55.633058 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:55.633516 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:55.633182 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:20:56.634946 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:56.634497 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:56.634946 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:56.634618 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:20:56.634946 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:56.634659 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:56.634946 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:56.634774 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:20:57.633009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:57.632978 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:57.633187 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:57.633098 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:20:58.633423 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:58.633239 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:20:58.633817 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:58.633326 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:20:58.633817 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:58.633519 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:20:58.633817 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:58.633614 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:20:58.724382 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:58.724349 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lxhnm" event={"ID":"926f2417-fb83-42f8-abee-55e41d925420","Type":"ContainerStarted","Data":"87b7bbf23e5a6dd35893f2e0b7e0d29ecbf51c52d2ee2b1326e15732b686c8d2"} Jul 16 12:20:58.726443 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:58.726419 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sw9kn" event={"ID":"7233e4f2-8303-45b4-8870-3ab003f5e5d8","Type":"ContainerStarted","Data":"0354d79e7fd158452d76822c182115e7820cac56bd5d50bfff1d89906311469c"} Jul 16 12:20:58.740672 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:58.740635 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-167.ec2.internal" podStartSLOduration=21.740623198 podStartE2EDuration="21.740623198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:20:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:20:41.700076398 +0000 UTC m=+5.561977644" watchObservedRunningTime="2026-07-16 12:20:58.740623198 +0000 UTC m=+22.602524444" Jul 16 12:20:58.740992 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:58.740969 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lxhnm" podStartSLOduration=2.594669457 podStartE2EDuration="21.740961909s" totalImagesPullingTime="19.146292452s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:20:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:20:58.740421499 +0000 UTC m=+22.602322744" watchObservedRunningTime="2026-07-16 12:20:58.740961909 +0000 UTC m=+22.602863155" Jul 16 12:20:58.757800 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:58.755963 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sw9kn" podStartSLOduration=3.641421625 podStartE2EDuration="22.755944846s" totalImagesPullingTime="19.114523221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:20:58.755239681 +0000 UTC m=+22.617140948" watchObservedRunningTime="2026-07-16 12:20:58.755944846 +0000 UTC m=+22.617846092" Jul 16 12:20:59.633024 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.632851 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:20:59.633201 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:20:59.633092 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:20:59.730343 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.730305 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mw9kk" event={"ID":"d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f","Type":"ContainerStarted","Data":"c3903034e018683e59aa1c6bd8d688c11cccc664cfca0b6eb845a533f0246042"} Jul 16 12:20:59.732995 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.732975 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:20:59.733286 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.733266 2577 generic.go:356] "Generic (PLEG): container finished" podID="13b833aa-75f5-490c-8df2-b9d2247c61d6" containerID="834e4653f1c058283794f772b6eaab300ebdd666027fe6dcf515f799d6b57040" exitCode=1 Jul 16 12:20:59.733355 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.733311 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" event={"ID":"13b833aa-75f5-490c-8df2-b9d2247c61d6","Type":"ContainerStarted","Data":"33919158fb60f64a8ac00d6ce45d4cb4dfba9f20f59bb1c10950aefd19eea02e"} Jul 16 12:20:59.733405 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.733365 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" event={"ID":"13b833aa-75f5-490c-8df2-b9d2247c61d6","Type":"ContainerStarted","Data":"0e2d2c7095387ea2b4c096c8654fcfe1e13a972ae5d13304039de5609df4d35e"} Jul 16 12:20:59.733405 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.733393 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" event={"ID":"13b833aa-75f5-490c-8df2-b9d2247c61d6","Type":"ContainerStarted","Data":"c63f3c3ca6a72a11d0ea4b88d67f56f025acab5546a8d9f85208da4dc594f694"} Jul 16 12:20:59.733504 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.733437 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" event={"ID":"13b833aa-75f5-490c-8df2-b9d2247c61d6","Type":"ContainerStarted","Data":"5ef8c560694aaffecbdc509d609f17875f5f6cfad85c5a809e230330d61d6444"} Jul 16 12:20:59.733504 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.733478 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" event={"ID":"13b833aa-75f5-490c-8df2-b9d2247c61d6","Type":"ContainerDied","Data":"834e4653f1c058283794f772b6eaab300ebdd666027fe6dcf515f799d6b57040"} Jul 16 12:20:59.733588 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.733513 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" event={"ID":"13b833aa-75f5-490c-8df2-b9d2247c61d6","Type":"ContainerStarted","Data":"77e73bd82b3fea2933de09aeadd99a1fc63dc38ceeaa22d66db631f30f21f633"} Jul 16 12:20:59.734660 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.734641 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" event={"ID":"d847d128-22b1-4fcf-a5c2-14d041546993","Type":"ContainerStarted","Data":"182b3ae9259a225a1a2c6031c1c598f61efb77e27626484614a4e63c88763f1c"} Jul 16 12:20:59.735924 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.735904 2577 generic.go:356] "Generic (PLEG): container finished" podID="5280509a-3dd9-492d-9eae-02e5c64d56e4" containerID="16dd05d8383a3239e7f49e48d988df68bcda6dc2c3a21aa4f15e229309b5dfb8" exitCode=0 Jul 16 12:20:59.735998 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.735978 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kxg9t" event={"ID":"5280509a-3dd9-492d-9eae-02e5c64d56e4","Type":"ContainerDied","Data":"16dd05d8383a3239e7f49e48d988df68bcda6dc2c3a21aa4f15e229309b5dfb8"} Jul 16 12:20:59.737391 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.737326 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" event={"ID":"f1a80591-5b2d-4604-9121-452bfcf30a3b","Type":"ContainerStarted","Data":"520007b3c7d5cf73a49739c81f289f33e2987b66f68ba643e07c706776ae15af"} Jul 16 12:20:59.738735 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.738714 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jqf5n" event={"ID":"95931072-3e08-422c-aff5-1e519977e11c","Type":"ContainerStarted","Data":"f3c46439ae164ef65b4c64808df6e5bec34ae6029dc5295325257e04ded771da"} Jul 16 12:20:59.749712 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.749677 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mw9kk" podStartSLOduration=4.610869715 podStartE2EDuration="23.749666014s" totalImagesPullingTime="19.138796299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:20:59.749388146 +0000 UTC m=+23.611289410" watchObservedRunningTime="2026-07-16 12:20:59.749666014 +0000 UTC m=+23.611567258" Jul 16 12:20:59.799086 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:20:59.799037 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5xbcz" podStartSLOduration=3.658626603 podStartE2EDuration="22.799025123s" totalImagesPullingTime="19.14039852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:20:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:20:59.798978787 +0000 UTC m=+23.660880031" watchObservedRunningTime="2026-07-16 12:20:59.799025123 +0000 UTC m=+23.660926367" Jul 16 12:21:00.065249 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:00.065113 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 12:21:00.538817 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:00.538703 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T12:21:00.065244421Z","UUID":"473c9c3d-a9df-4ca3-91de-2b0b6f29b2a4","Handler":null,"Name":"","Endpoint":""} Jul 16 12:21:00.540053 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:00.540038 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 12:21:00.540145 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:00.540072 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 12:21:00.633140 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:00.633106 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:00.633343 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:00.633253 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:00.633343 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:00.633322 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:00.633473 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:00.633433 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:00.743005 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:00.742966 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9f5jm" event={"ID":"a99347fd-ae73-45ca-942e-9d5d91d59c2a","Type":"ContainerStarted","Data":"9e8fcf6dd264d216de15234ac5bba106b84b6252d87d32c62b86e3ff94bf80b4"} Jul 16 12:21:00.745931 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:00.745506 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" event={"ID":"d847d128-22b1-4fcf-a5c2-14d041546993","Type":"ContainerStarted","Data":"6062d76b994d7db17db01950fd7a0256c76f4ec78322dde59f0b7574818bfd57"} Jul 16 12:21:00.760187 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:00.760149 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jqf5n" podStartSLOduration=5.22498426 podStartE2EDuration="24.760135916s" totalImagesPullingTime="19.535151656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:20:59.819047194 +0000 UTC m=+23.680948450" watchObservedRunningTime="2026-07-16 12:21:00.760135916 +0000 UTC m=+24.622037207" Jul 16 12:21:00.760513 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:00.760489 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9f5jm" podStartSLOduration=5.652373861 podStartE2EDuration="24.76048383s" totalImagesPullingTime="19.108109969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:00.759885452 +0000 UTC m=+24.621786698" watchObservedRunningTime="2026-07-16 12:21:00.76048383 +0000 UTC m=+24.622385105" Jul 16 12:21:01.633632 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:01.633550 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:01.633781 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:01.633675 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:01.750927 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:01.750902 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:21:01.754710 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:01.754679 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" event={"ID":"13b833aa-75f5-490c-8df2-b9d2247c61d6","Type":"ContainerStarted","Data":"22f28267c589ce119996f2007396366c08b7640af689a35416481c110a430b3d"} Jul 16 12:21:01.757136 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:01.757070 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" event={"ID":"d847d128-22b1-4fcf-a5c2-14d041546993","Type":"ContainerStarted","Data":"22dd9c93dfbbf42a51b9f18b0ea458f02302d840a13ad5db7631da07dce2904e"} Jul 16 12:21:01.777220 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:01.777176 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dmgzf" podStartSLOduration=4.369835685 podStartE2EDuration="25.777160769s" totalImagesPullingTime="21.407325084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:20:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:01.776679558 +0000 UTC m=+25.638580803" watchObservedRunningTime="2026-07-16 12:21:01.777160769 +0000 UTC m=+25.639062035" Jul 16 12:21:02.633257 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:02.633222 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:02.633457 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:02.633354 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:02.633457 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:02.633364 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:02.633561 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:02.633464 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:03.093689 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:03.093659 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sw9kn" Jul 16 12:21:03.094664 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:03.094644 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sw9kn" Jul 16 12:21:03.633173 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:03.633138 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:03.633349 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:03.633261 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:04.633661 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:04.633627 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:04.634166 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:04.633769 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:04.634166 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:04.633828 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:04.634166 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:04.633939 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:05.467074 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:05.466844 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sw9kn" Jul 16 12:21:05.467189 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:05.467170 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 16 12:21:05.467599 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:05.467586 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sw9kn" Jul 16 12:21:05.633287 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:05.633258 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:05.633458 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:05.633388 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:05.769060 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:05.768991 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:21:05.769575 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:05.769412 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" event={"ID":"13b833aa-75f5-490c-8df2-b9d2247c61d6","Type":"ContainerStarted","Data":"e0dbaad12bef38ee72c6da3d13639705fcdebed23595197f36e56e2b7127242d"} Jul 16 12:21:05.769708 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:05.769693 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:21:05.769772 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:05.769746 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:21:05.769866 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:05.769849 2577 scope.go:117] "RemoveContainer" containerID="834e4653f1c058283794f772b6eaab300ebdd666027fe6dcf515f799d6b57040" Jul 16 12:21:05.771347 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:05.771323 2577 generic.go:356] "Generic (PLEG): container finished" podID="5280509a-3dd9-492d-9eae-02e5c64d56e4" containerID="752076f1db1e08170495afb86399afe889750fd3754bd861e40ea743f1d43143" exitCode=0 Jul 16 12:21:05.771448 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:05.771409 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kxg9t" event={"ID":"5280509a-3dd9-492d-9eae-02e5c64d56e4","Type":"ContainerDied","Data":"752076f1db1e08170495afb86399afe889750fd3754bd861e40ea743f1d43143"} Jul 16 12:21:05.787199 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:05.787182 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:21:06.634048 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:06.633889 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:06.634326 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:06.633958 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:06.634326 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:06.634105 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:06.634326 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:06.634204 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:06.776666 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:06.776637 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:21:06.777075 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:06.776929 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" event={"ID":"13b833aa-75f5-490c-8df2-b9d2247c61d6","Type":"ContainerStarted","Data":"bee3bc368aa917b0549cc54635239f96470ddf435ee5618a765154579145bcd5"} Jul 16 12:21:06.777150 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:06.777133 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:21:06.792312 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:06.792277 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:21:06.808221 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:06.808170 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" podStartSLOduration=11.631388377 podStartE2EDuration="30.808156657s" totalImagesPullingTime="19.17676828s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:06.807667939 +0000 UTC m=+30.669569202" watchObservedRunningTime="2026-07-16 12:21:06.808156657 +0000 UTC m=+30.670057901" Jul 16 12:21:07.633136 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:07.633104 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:07.633333 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:07.633202 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:07.780874 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:07.780834 2577 generic.go:356] "Generic (PLEG): container finished" podID="5280509a-3dd9-492d-9eae-02e5c64d56e4" containerID="d2c816eb0dc030f257e1dc46dbaf91d4ff403088d5ff797788f1387c018e09b8" exitCode=0 Jul 16 12:21:07.781261 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:07.780901 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kxg9t" event={"ID":"5280509a-3dd9-492d-9eae-02e5c64d56e4","Type":"ContainerDied","Data":"d2c816eb0dc030f257e1dc46dbaf91d4ff403088d5ff797788f1387c018e09b8"} Jul 16 12:21:08.633457 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:08.633423 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:08.633626 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:08.633570 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:08.633626 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:08.633616 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:08.633721 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:08.633700 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:09.633479 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:09.633442 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:09.633955 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:09.633579 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:10.228579 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:10.228550 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs\") pod \"network-metrics-daemon-98bqc\" (UID: \"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e\") " pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:10.228698 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:10.228671 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:21:10.228747 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:10.228719 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs podName:bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e nodeName:}" failed. No retries permitted until 2026-07-16 12:21:42.228706338 +0000 UTC m=+66.090607560 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs") pod "network-metrics-daemon-98bqc" (UID: "bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 12:21:10.329474 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:10.329390 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fjv2\" (UniqueName: \"kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2\") pod \"network-check-target-4qrhp\" (UID: \"ac6ef832-3b6b-4b74-a5bf-3823389a6d1c\") " pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:10.329474 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:10.329433 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:10.329674 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:10.329527 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 12:21:10.329674 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:10.329555 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 12:21:10.329674 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:10.329576 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 12:21:10.329674 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:10.329580 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret podName:4a3e09d3-661a-4a7f-8741-f39758ea7303 nodeName:}" failed. No retries permitted until 2026-07-16 12:21:42.329566698 +0000 UTC m=+66.191467920 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret") pod "global-pull-secret-syncer-wjdcb" (UID: "4a3e09d3-661a-4a7f-8741-f39758ea7303") : object "kube-system"/"original-pull-secret" not registered Jul 16 12:21:10.329674 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:10.329588 2577 projected.go:196] Error preparing data for projected volume kube-api-access-6fjv2 for pod openshift-network-diagnostics/network-check-target-4qrhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:21:10.329674 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:10.329637 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2 podName:ac6ef832-3b6b-4b74-a5bf-3823389a6d1c nodeName:}" failed. No retries permitted until 2026-07-16 12:21:42.329621159 +0000 UTC m=+66.191522384 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6fjv2" (UniqueName: "kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2") pod "network-check-target-4qrhp" (UID: "ac6ef832-3b6b-4b74-a5bf-3823389a6d1c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 12:21:10.633604 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:10.633533 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:10.633942 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:10.633533 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:10.633942 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:10.633649 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:10.633942 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:10.633696 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:10.789094 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:10.789063 2577 generic.go:356] "Generic (PLEG): container finished" podID="5280509a-3dd9-492d-9eae-02e5c64d56e4" containerID="ecada85090a2a891d013361f72b5a86fb0d85b2a28fd19c1b73e8b2a46e8da70" exitCode=0 Jul 16 12:21:10.789260 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:10.789129 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kxg9t" event={"ID":"5280509a-3dd9-492d-9eae-02e5c64d56e4","Type":"ContainerDied","Data":"ecada85090a2a891d013361f72b5a86fb0d85b2a28fd19c1b73e8b2a46e8da70"} Jul 16 12:21:11.633420 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:11.633382 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:11.633609 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:11.633501 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:12.632924 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:12.632857 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:12.633105 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:12.633001 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:12.633169 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:12.633095 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:12.633223 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:12.633193 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:13.633593 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:13.633562 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:13.634027 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:13.633695 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:14.633206 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:14.633176 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:14.633407 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:14.633176 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:14.633407 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:14.633327 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:14.633507 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:14.633404 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:15.633656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:15.633625 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:15.634091 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:15.633727 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:15.804860 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:15.804831 2577 generic.go:356] "Generic (PLEG): container finished" podID="5280509a-3dd9-492d-9eae-02e5c64d56e4" containerID="00555b0763ceca9c78345d3d62c6c4dc17a04f8cf21ae155d14eef19dde4dd58" exitCode=0 Jul 16 12:21:15.805016 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:15.804889 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kxg9t" event={"ID":"5280509a-3dd9-492d-9eae-02e5c64d56e4","Type":"ContainerDied","Data":"00555b0763ceca9c78345d3d62c6c4dc17a04f8cf21ae155d14eef19dde4dd58"} Jul 16 12:21:16.634332 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:16.634279 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:16.634791 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:16.634390 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:16.634791 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:16.634446 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:16.634791 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:16.634524 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:16.810169 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:16.810139 2577 generic.go:356] "Generic (PLEG): container finished" podID="5280509a-3dd9-492d-9eae-02e5c64d56e4" containerID="f54a27959a7e63ba90b720eeac8176dc539de098283b424074151ee187851d30" exitCode=0 Jul 16 12:21:16.810323 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:16.810179 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kxg9t" event={"ID":"5280509a-3dd9-492d-9eae-02e5c64d56e4","Type":"ContainerDied","Data":"f54a27959a7e63ba90b720eeac8176dc539de098283b424074151ee187851d30"} Jul 16 12:21:17.633082 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:17.633053 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:17.633243 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:17.633145 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:17.815225 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:17.815188 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kxg9t" event={"ID":"5280509a-3dd9-492d-9eae-02e5c64d56e4","Type":"ContainerStarted","Data":"2867df6ef5bc4153722e3619882b943ed21ca85699c0648d539c1b2913444396"} Jul 16 12:21:17.837347 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:17.837287 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kxg9t" podStartSLOduration=9.352335095 podStartE2EDuration="41.837272003s" totalImagesPullingTime="32.484936908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:20:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:17.835824762 +0000 UTC m=+41.697726017" watchObservedRunningTime="2026-07-16 12:21:17.837272003 +0000 UTC m=+41.699173248" Jul 16 12:21:18.633122 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:18.633093 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:18.633284 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:18.633098 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:18.633284 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:18.633192 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:18.633398 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:18.633322 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:19.633621 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:19.633596 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:19.633980 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:19.633687 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:20.633533 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:20.633494 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:20.633706 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:20.633610 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:20.633706 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:20.633658 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:20.634277 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:20.633738 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:21.633543 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:21.633513 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:21.633705 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:21.633608 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:22.636371 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:22.636344 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:22.636851 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:22.636344 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:22.636851 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:22.636434 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:22.636851 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:22.636529 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:23.633270 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:23.633240 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:23.633430 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:23.633360 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:24.059244 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:24.059045 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-98bqc"] Jul 16 12:21:24.059693 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:24.059364 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:24.059693 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:24.059454 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:24.061953 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:24.061928 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wjdcb"] Jul 16 12:21:24.062082 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:24.062015 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:24.062142 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:24.062095 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:24.062566 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:24.062540 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4qrhp"] Jul 16 12:21:24.062676 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:24.062625 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:24.062728 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:24.062691 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:25.632763 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:25.632683 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:25.633144 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:25.632783 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:25.633144 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:25.632683 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:25.633144 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:25.632852 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:25.633144 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:25.632683 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:25.633144 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:25.632910 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:27.633747 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:27.633713 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:27.634181 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:27.633816 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:27.634181 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:27.633824 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wjdcb" podUID="4a3e09d3-661a-4a7f-8741-f39758ea7303" Jul 16 12:21:27.634181 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:27.633849 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:27.634181 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:27.633935 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-98bqc" podUID="bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e" Jul 16 12:21:27.634181 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:27.633991 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4qrhp" podUID="ac6ef832-3b6b-4b74-a5bf-3823389a6d1c" Jul 16 12:21:28.466042 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.466017 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-167.ec2.internal" event="NodeReady" Jul 16 12:21:28.466162 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.466113 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 12:21:28.509228 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.509161 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75f6d5c9cd-ssrzn"] Jul 16 12:21:28.534147 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.534126 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.537277 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.537249 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75f6d5c9cd-ssrzn"] Jul 16 12:21:28.538002 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.537974 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 12:21:28.538091 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.538019 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 12:21:28.538520 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.538481 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-drrn2\"" Jul 16 12:21:28.538594 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.538491 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 12:21:28.543418 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.543390 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 12:21:28.548220 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.548202 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-99pq4"] Jul 16 12:21:28.560795 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.560774 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-trusted-ca\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.560860 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.560802 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-registry-tls\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.560860 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.560819 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-ca-trust-extracted\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.560860 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.560848 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-registry-certificates\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.560962 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.560883 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gpq5\" (UniqueName: \"kubernetes.io/projected/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-kube-api-access-6gpq5\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.560962 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.560931 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-installation-pull-secrets\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.560962 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.560956 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-image-registry-private-configuration\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.561052 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.560974 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-bound-sa-token\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.574152 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.574129 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xgspb"] Jul 16 12:21:28.574334 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.574243 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.577886 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.577401 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 12:21:28.578081 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.577996 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 12:21:28.579628 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.578563 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 12:21:28.579628 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.578670 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 12:21:28.579628 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.578786 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5tkvf\"" Jul 16 12:21:28.600259 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.600228 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xgspb"] Jul 16 12:21:28.600356 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.600269 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-99pq4"] Jul 16 12:21:28.600356 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.600348 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.602850 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.602835 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 12:21:28.603108 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.603093 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 12:21:28.614082 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.614067 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-drvvw\"" Jul 16 12:21:28.627403 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.627377 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hzjdc"] Jul 16 12:21:28.638513 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.638489 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hzjdc" Jul 16 12:21:28.641985 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.641964 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 12:21:28.642072 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.642009 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 12:21:28.642151 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.642076 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 12:21:28.642236 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.642221 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-khcsh\"" Jul 16 12:21:28.643034 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.643010 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hzjdc"] Jul 16 12:21:28.661338 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661317 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-bound-sa-token\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.661442 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661355 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f283e409-c6e8-4dde-9bbb-4190973d2321-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.661442 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661382 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f283e409-c6e8-4dde-9bbb-4190973d2321-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.661442 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-trusted-ca\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.661604 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661447 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-registry-tls\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.661604 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661474 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-ca-trust-extracted\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.661604 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661502 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-registry-certificates\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.661745 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661688 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86b7ee17-6296-45e8-8f98-f6ca03f75c73-tmp-dir\") pod \"dns-default-xgspb\" (UID: \"86b7ee17-6296-45e8-8f98-f6ca03f75c73\") " pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.661796 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661757 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gpq5\" (UniqueName: \"kubernetes.io/projected/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-kube-api-access-6gpq5\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.661851 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661793 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/86b7ee17-6296-45e8-8f98-f6ca03f75c73-config-volume\") pod \"dns-default-xgspb\" (UID: \"86b7ee17-6296-45e8-8f98-f6ca03f75c73\") " pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.661851 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661829 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/86b7ee17-6296-45e8-8f98-f6ca03f75c73-metrics-tls\") pod \"dns-default-xgspb\" (UID: \"86b7ee17-6296-45e8-8f98-f6ca03f75c73\") " pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.661948 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661882 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbmvp\" (UniqueName: \"kubernetes.io/projected/f283e409-c6e8-4dde-9bbb-4190973d2321-kube-api-access-bbmvp\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.661948 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661903 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-ca-trust-extracted\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.661948 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661915 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qgpq\" (UniqueName: \"kubernetes.io/projected/b8ea5963-0f56-4b84-a76e-c53646bf650d-kube-api-access-9qgpq\") pod \"ingress-canary-hzjdc\" (UID: \"b8ea5963-0f56-4b84-a76e-c53646bf650d\") " pod="openshift-ingress-canary/ingress-canary-hzjdc" Jul 16 12:21:28.662088 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661956 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs5nx\" (UniqueName: \"kubernetes.io/projected/86b7ee17-6296-45e8-8f98-f6ca03f75c73-kube-api-access-rs5nx\") pod \"dns-default-xgspb\" (UID: \"86b7ee17-6296-45e8-8f98-f6ca03f75c73\") " pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.662088 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.661985 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f283e409-c6e8-4dde-9bbb-4190973d2321-data-volume\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.662088 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.662008 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8ea5963-0f56-4b84-a76e-c53646bf650d-cert\") pod \"ingress-canary-hzjdc\" (UID: \"b8ea5963-0f56-4b84-a76e-c53646bf650d\") " pod="openshift-ingress-canary/ingress-canary-hzjdc" Jul 16 12:21:28.662088 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.662039 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-installation-pull-secrets\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.662088 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.662062 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f283e409-c6e8-4dde-9bbb-4190973d2321-crio-socket\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.662358 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.662104 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-image-registry-private-configuration\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.662358 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.662266 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-registry-certificates\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.662579 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.662553 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-trusted-ca\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.665968 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.665939 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-installation-pull-secrets\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.666048 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.665949 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-image-registry-private-configuration\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.666048 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.665986 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-registry-tls\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.675115 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.675094 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gpq5\" (UniqueName: \"kubernetes.io/projected/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-kube-api-access-6gpq5\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.675207 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.675193 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f36b38c2-0c06-4df7-8d87-3c574fdeaf31-bound-sa-token\") pod \"image-registry-75f6d5c9cd-ssrzn\" (UID: \"f36b38c2-0c06-4df7-8d87-3c574fdeaf31\") " pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.763400 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.763336 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86b7ee17-6296-45e8-8f98-f6ca03f75c73-tmp-dir\") pod \"dns-default-xgspb\" (UID: \"86b7ee17-6296-45e8-8f98-f6ca03f75c73\") " pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.763400 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.763372 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/86b7ee17-6296-45e8-8f98-f6ca03f75c73-config-volume\") pod \"dns-default-xgspb\" (UID: \"86b7ee17-6296-45e8-8f98-f6ca03f75c73\") " pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.763400 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.763389 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/86b7ee17-6296-45e8-8f98-f6ca03f75c73-metrics-tls\") pod \"dns-default-xgspb\" (UID: \"86b7ee17-6296-45e8-8f98-f6ca03f75c73\") " pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.763691 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.763542 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbmvp\" (UniqueName: \"kubernetes.io/projected/f283e409-c6e8-4dde-9bbb-4190973d2321-kube-api-access-bbmvp\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.763691 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.763579 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qgpq\" (UniqueName: \"kubernetes.io/projected/b8ea5963-0f56-4b84-a76e-c53646bf650d-kube-api-access-9qgpq\") pod \"ingress-canary-hzjdc\" (UID: \"b8ea5963-0f56-4b84-a76e-c53646bf650d\") " pod="openshift-ingress-canary/ingress-canary-hzjdc" Jul 16 12:21:28.763691 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.763601 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs5nx\" (UniqueName: \"kubernetes.io/projected/86b7ee17-6296-45e8-8f98-f6ca03f75c73-kube-api-access-rs5nx\") pod \"dns-default-xgspb\" (UID: \"86b7ee17-6296-45e8-8f98-f6ca03f75c73\") " pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.763691 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.763628 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f283e409-c6e8-4dde-9bbb-4190973d2321-data-volume\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.763691 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.763667 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8ea5963-0f56-4b84-a76e-c53646bf650d-cert\") pod \"ingress-canary-hzjdc\" (UID: \"b8ea5963-0f56-4b84-a76e-c53646bf650d\") " pod="openshift-ingress-canary/ingress-canary-hzjdc" Jul 16 12:21:28.763691 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.763675 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86b7ee17-6296-45e8-8f98-f6ca03f75c73-tmp-dir\") pod \"dns-default-xgspb\" (UID: \"86b7ee17-6296-45e8-8f98-f6ca03f75c73\") " pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.764184 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.763993 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f283e409-c6e8-4dde-9bbb-4190973d2321-crio-socket\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.764184 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.764014 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/86b7ee17-6296-45e8-8f98-f6ca03f75c73-config-volume\") pod \"dns-default-xgspb\" (UID: \"86b7ee17-6296-45e8-8f98-f6ca03f75c73\") " pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.764184 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.764018 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f283e409-c6e8-4dde-9bbb-4190973d2321-data-volume\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.764184 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.764050 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f283e409-c6e8-4dde-9bbb-4190973d2321-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.764184 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.764086 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f283e409-c6e8-4dde-9bbb-4190973d2321-crio-socket\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.764184 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.764086 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f283e409-c6e8-4dde-9bbb-4190973d2321-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.764612 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.764579 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f283e409-c6e8-4dde-9bbb-4190973d2321-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.765795 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.765766 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/86b7ee17-6296-45e8-8f98-f6ca03f75c73-metrics-tls\") pod \"dns-default-xgspb\" (UID: \"86b7ee17-6296-45e8-8f98-f6ca03f75c73\") " pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.766248 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.766224 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8ea5963-0f56-4b84-a76e-c53646bf650d-cert\") pod \"ingress-canary-hzjdc\" (UID: \"b8ea5963-0f56-4b84-a76e-c53646bf650d\") " pod="openshift-ingress-canary/ingress-canary-hzjdc" Jul 16 12:21:28.766368 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.766332 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f283e409-c6e8-4dde-9bbb-4190973d2321-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.772991 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.772963 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbmvp\" (UniqueName: \"kubernetes.io/projected/f283e409-c6e8-4dde-9bbb-4190973d2321-kube-api-access-bbmvp\") pod \"insights-runtime-extractor-99pq4\" (UID: \"f283e409-c6e8-4dde-9bbb-4190973d2321\") " pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.773240 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.773220 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qgpq\" (UniqueName: \"kubernetes.io/projected/b8ea5963-0f56-4b84-a76e-c53646bf650d-kube-api-access-9qgpq\") pod \"ingress-canary-hzjdc\" (UID: \"b8ea5963-0f56-4b84-a76e-c53646bf650d\") " pod="openshift-ingress-canary/ingress-canary-hzjdc" Jul 16 12:21:28.773340 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.773328 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs5nx\" (UniqueName: \"kubernetes.io/projected/86b7ee17-6296-45e8-8f98-f6ca03f75c73-kube-api-access-rs5nx\") pod \"dns-default-xgspb\" (UID: \"86b7ee17-6296-45e8-8f98-f6ca03f75c73\") " pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.844561 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.844523 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:28.887662 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.887628 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-99pq4" Jul 16 12:21:28.907582 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.907555 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:28.956494 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:28.955882 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hzjdc" Jul 16 12:21:29.063787 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.056358 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-99pq4"] Jul 16 12:21:29.082829 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.082753 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75f6d5c9cd-ssrzn"] Jul 16 12:21:29.099363 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.099281 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xgspb"] Jul 16 12:21:29.104574 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:21:29.104547 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86b7ee17_6296_45e8_8f98_f6ca03f75c73.slice/crio-be34cb2c3d540bd153cfc2e7c74fb89ec0f022e4a4eaf11d1ba2bec75bd3e515 WatchSource:0}: Error finding container be34cb2c3d540bd153cfc2e7c74fb89ec0f022e4a4eaf11d1ba2bec75bd3e515: Status 404 returned error can't find the container with id be34cb2c3d540bd153cfc2e7c74fb89ec0f022e4a4eaf11d1ba2bec75bd3e515 Jul 16 12:21:29.143471 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.143444 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hzjdc"] Jul 16 12:21:29.155218 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:21:29.155180 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ea5963_0f56_4b84_a76e_c53646bf650d.slice/crio-c3872d6b48a775cf1783fb6b7432d320846f9c320f94a16ed408c4f0948105b8 WatchSource:0}: Error finding container c3872d6b48a775cf1783fb6b7432d320846f9c320f94a16ed408c4f0948105b8: Status 404 returned error can't find the container with id c3872d6b48a775cf1783fb6b7432d320846f9c320f94a16ed408c4f0948105b8 Jul 16 12:21:29.632793 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.632758 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:29.632953 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.632804 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:29.632953 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.632758 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:29.636782 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.636749 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 12:21:29.637044 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.637028 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 12:21:29.638027 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.637266 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6rr4c\"" Jul 16 12:21:29.638027 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.637510 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 12:21:29.638027 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.637755 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 12:21:29.638027 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.638003 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qp65q\"" Jul 16 12:21:29.845285 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.845246 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xgspb" event={"ID":"86b7ee17-6296-45e8-8f98-f6ca03f75c73","Type":"ContainerStarted","Data":"be34cb2c3d540bd153cfc2e7c74fb89ec0f022e4a4eaf11d1ba2bec75bd3e515"} Jul 16 12:21:29.846752 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.846619 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hzjdc" event={"ID":"b8ea5963-0f56-4b84-a76e-c53646bf650d","Type":"ContainerStarted","Data":"c3872d6b48a775cf1783fb6b7432d320846f9c320f94a16ed408c4f0948105b8"} Jul 16 12:21:29.848091 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.848062 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-99pq4" event={"ID":"f283e409-c6e8-4dde-9bbb-4190973d2321","Type":"ContainerStarted","Data":"c120977ce522588ab383a7140c30232fb3e722956b3412fd17a8c79ff071d7c0"} Jul 16 12:21:29.849877 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.849850 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" event={"ID":"f36b38c2-0c06-4df7-8d87-3c574fdeaf31","Type":"ContainerStarted","Data":"c5a35ce4d4f157b35ce396421ebff58f4f147fb0eddf679d2fe1cfd95cdf8c31"} Jul 16 12:21:29.849971 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.849899 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" event={"ID":"f36b38c2-0c06-4df7-8d87-3c574fdeaf31","Type":"ContainerStarted","Data":"853eb9f584c6b62d47ab447e261e37a54142acdea5bbac3ff7ecaea7fcfa8819"} Jul 16 12:21:29.850032 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:29.850015 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:31.857280 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:31.857021 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xgspb" event={"ID":"86b7ee17-6296-45e8-8f98-f6ca03f75c73","Type":"ContainerStarted","Data":"ec63cdf26ee8462f237f2d012c6dfcfe848e4768c9118c9da1306ca35d334b40"} Jul 16 12:21:31.857711 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:31.857344 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:31.857711 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:31.857373 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xgspb" event={"ID":"86b7ee17-6296-45e8-8f98-f6ca03f75c73","Type":"ContainerStarted","Data":"6963dd909e2775708f328274ad84ae265bc838d476b830a54cdfe185bdc76a6d"} Jul 16 12:21:31.877740 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:31.877684 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" podStartSLOduration=8.877666475 podStartE2EDuration="8.877666475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:21:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:29.876527744 +0000 UTC m=+53.738429007" watchObservedRunningTime="2026-07-16 12:21:31.877666475 +0000 UTC m=+55.739567720" Jul 16 12:21:32.862692 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:32.862598 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hzjdc" event={"ID":"b8ea5963-0f56-4b84-a76e-c53646bf650d","Type":"ContainerStarted","Data":"ed009b79ab1e7f35133abe633339f71fabf59a265c476c7a08705ba83e7561c3"} Jul 16 12:21:32.879538 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:32.879491 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xgspb" podStartSLOduration=2.70550997 podStartE2EDuration="4.879475263s" totalImagesPullingTime="2.173965293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:31.878006424 +0000 UTC m=+55.739907670" watchObservedRunningTime="2026-07-16 12:21:32.879475263 +0000 UTC m=+56.741376487" Jul 16 12:21:32.880479 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:32.880438 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hzjdc" podStartSLOduration=1.63688037 podStartE2EDuration="4.880422523s" totalImagesPullingTime="3.243542153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:32.879442071 +0000 UTC m=+56.741343316" watchObservedRunningTime="2026-07-16 12:21:32.880422523 +0000 UTC m=+56.742323773" Jul 16 12:21:35.872501 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:35.872469 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-99pq4" event={"ID":"f283e409-c6e8-4dde-9bbb-4190973d2321","Type":"ContainerStarted","Data":"70d36ffd66dc616d051ec0dbd3cfc9fb421e5eed838623744df62ac3162dc681"} Jul 16 12:21:38.794734 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:38.794707 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7cwqf" Jul 16 12:21:39.887426 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:39.887388 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-99pq4" event={"ID":"f283e409-c6e8-4dde-9bbb-4190973d2321","Type":"ContainerStarted","Data":"75c311fa4dc20f9407cf62af174464b5ec0cb7f9f308545ca6074ddf10220255"} Jul 16 12:21:41.864910 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:41.864883 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xgspb" Jul 16 12:21:42.280316 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.280253 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs\") pod \"network-metrics-daemon-98bqc\" (UID: \"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e\") " pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:42.282824 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.282803 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 12:21:42.295445 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.295425 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e-metrics-certs\") pod \"network-metrics-daemon-98bqc\" (UID: \"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e\") " pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:42.380803 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.380756 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:42.380987 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.380855 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fjv2\" (UniqueName: \"kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2\") pod \"network-check-target-4qrhp\" (UID: \"ac6ef832-3b6b-4b74-a5bf-3823389a6d1c\") " pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:42.383605 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.383585 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 12:21:42.383724 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.383612 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 12:21:42.396134 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.396112 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 12:21:42.396260 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.396216 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a3e09d3-661a-4a7f-8741-f39758ea7303-original-pull-secret\") pod \"global-pull-secret-syncer-wjdcb\" (UID: \"4a3e09d3-661a-4a7f-8741-f39758ea7303\") " pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:42.405165 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.405143 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fjv2\" (UniqueName: \"kubernetes.io/projected/ac6ef832-3b6b-4b74-a5bf-3823389a6d1c-kube-api-access-6fjv2\") pod \"network-check-target-4qrhp\" (UID: \"ac6ef832-3b6b-4b74-a5bf-3823389a6d1c\") " pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:42.551468 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.551392 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6rr4c\"" Jul 16 12:21:42.556468 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.556443 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wjdcb" Jul 16 12:21:42.559592 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.559551 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-98bqc" Jul 16 12:21:42.565081 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.565064 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qp65q\"" Jul 16 12:21:42.573177 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.573160 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:42.833879 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.833850 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-98bqc"] Jul 16 12:21:42.836947 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:21:42.836920 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6fd46f_f94f_4f9f_8aca_89a63ab2a44e.slice/crio-a988dda7d39832131344a492d1aa048d38a06ae13697cd16a8eaf62b20d6e62a WatchSource:0}: Error finding container a988dda7d39832131344a492d1aa048d38a06ae13697cd16a8eaf62b20d6e62a: Status 404 returned error can't find the container with id a988dda7d39832131344a492d1aa048d38a06ae13697cd16a8eaf62b20d6e62a Jul 16 12:21:42.899405 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.899369 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-99pq4" event={"ID":"f283e409-c6e8-4dde-9bbb-4190973d2321","Type":"ContainerStarted","Data":"25eb540fee7581dd4b2bb27ff364e7a50110f13e582496ca3a1e0278d3d71193"} Jul 16 12:21:42.900565 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.900531 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-98bqc" event={"ID":"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e","Type":"ContainerStarted","Data":"a988dda7d39832131344a492d1aa048d38a06ae13697cd16a8eaf62b20d6e62a"} Jul 16 12:21:42.916993 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:42.916956 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-99pq4" podStartSLOduration=1.35713943 podStartE2EDuration="14.916940423s" totalImagesPullingTime="13.559800993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:21:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:42.916691722 +0000 UTC m=+66.778592980" watchObservedRunningTime="2026-07-16 12:21:42.916940423 +0000 UTC m=+66.778841645" Jul 16 12:21:43.054116 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.054082 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wjdcb"] Jul 16 12:21:43.057133 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:21:43.057108 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a3e09d3_661a_4a7f_8741_f39758ea7303.slice/crio-c110e683ab2fe4e9af9e89db44cb6609023069938bd352f2af4d88e7edb0be87 WatchSource:0}: Error finding container c110e683ab2fe4e9af9e89db44cb6609023069938bd352f2af4d88e7edb0be87: Status 404 returned error can't find the container with id c110e683ab2fe4e9af9e89db44cb6609023069938bd352f2af4d88e7edb0be87 Jul 16 12:21:43.059958 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.059939 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4qrhp"] Jul 16 12:21:43.062377 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:21:43.062358 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6ef832_3b6b_4b74_a5bf_3823389a6d1c.slice/crio-089721092e96623b653f4da3d6a3c9b29d7090408f2cf4d94f562bc56afd4713 WatchSource:0}: Error finding container 089721092e96623b653f4da3d6a3c9b29d7090408f2cf4d94f562bc56afd4713: Status 404 returned error can't find the container with id 089721092e96623b653f4da3d6a3c9b29d7090408f2cf4d94f562bc56afd4713 Jul 16 12:21:43.748260 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.748210 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hg7qx"] Jul 16 12:21:43.751889 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.751870 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.755001 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.754827 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 12:21:43.755001 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.754920 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 12:21:43.755884 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.754941 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 12:21:43.755974 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.755896 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5n6tr\"" Jul 16 12:21:43.756114 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.756094 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 12:21:43.756114 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.756111 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 12:21:43.756519 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.756496 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 12:21:43.790437 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.790405 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-textfile\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.790584 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.790457 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/029311a2-84cd-4e64-81c5-65c0b73d9aad-root\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.790584 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.790498 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-accelerators-collector-config\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.790584 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.790520 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/029311a2-84cd-4e64-81c5-65c0b73d9aad-sys\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.790584 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.790545 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/029311a2-84cd-4e64-81c5-65c0b73d9aad-metrics-client-ca\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.790782 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.790588 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-tls\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.790782 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.790611 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-wtmp\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.790782 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.790636 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.790782 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.790665 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqfhm\" (UniqueName: \"kubernetes.io/projected/029311a2-84cd-4e64-81c5-65c0b73d9aad-kube-api-access-sqfhm\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.891274 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.891246 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqfhm\" (UniqueName: \"kubernetes.io/projected/029311a2-84cd-4e64-81c5-65c0b73d9aad-kube-api-access-sqfhm\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.891380 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.891312 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-textfile\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.891380 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.891346 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/029311a2-84cd-4e64-81c5-65c0b73d9aad-root\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.891380 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.891366 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-accelerators-collector-config\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.891532 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.891385 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/029311a2-84cd-4e64-81c5-65c0b73d9aad-sys\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.891532 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.891411 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/029311a2-84cd-4e64-81c5-65c0b73d9aad-metrics-client-ca\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.891532 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.891457 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-tls\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.891532 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.891485 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-wtmp\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.891532 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.891504 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.891777 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.891671 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/029311a2-84cd-4e64-81c5-65c0b73d9aad-sys\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.892014 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.891988 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-textfile\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.892123 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.891990 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/029311a2-84cd-4e64-81c5-65c0b73d9aad-root\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.892123 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.892102 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-wtmp\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.892491 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.892465 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/029311a2-84cd-4e64-81c5-65c0b73d9aad-metrics-client-ca\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.892592 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.892558 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-accelerators-collector-config\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.894141 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.894122 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.894424 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.894404 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/029311a2-84cd-4e64-81c5-65c0b73d9aad-node-exporter-tls\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.902323 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.902259 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqfhm\" (UniqueName: \"kubernetes.io/projected/029311a2-84cd-4e64-81c5-65c0b73d9aad-kube-api-access-sqfhm\") pod \"node-exporter-hg7qx\" (UID: \"029311a2-84cd-4e64-81c5-65c0b73d9aad\") " pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:43.906106 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.906085 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4qrhp" event={"ID":"ac6ef832-3b6b-4b74-a5bf-3823389a6d1c","Type":"ContainerStarted","Data":"089721092e96623b653f4da3d6a3c9b29d7090408f2cf4d94f562bc56afd4713"} Jul 16 12:21:43.907653 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:43.907622 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wjdcb" event={"ID":"4a3e09d3-661a-4a7f-8741-f39758ea7303","Type":"ContainerStarted","Data":"c110e683ab2fe4e9af9e89db44cb6609023069938bd352f2af4d88e7edb0be87"} Jul 16 12:21:44.063844 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:44.063812 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hg7qx" Jul 16 12:21:44.076839 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:21:44.076807 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029311a2_84cd_4e64_81c5_65c0b73d9aad.slice/crio-2bd1f06a410009fe7a688061145446adc1c3a936e63c0f79d2bd41938b9e1df9 WatchSource:0}: Error finding container 2bd1f06a410009fe7a688061145446adc1c3a936e63c0f79d2bd41938b9e1df9: Status 404 returned error can't find the container with id 2bd1f06a410009fe7a688061145446adc1c3a936e63c0f79d2bd41938b9e1df9 Jul 16 12:21:44.914304 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:44.914247 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-98bqc" event={"ID":"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e","Type":"ContainerStarted","Data":"f945ea8a7e827f6754aa7ed2e505faf66fb8ff6d151265e72a18772dfbbf1594"} Jul 16 12:21:44.914736 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:44.914332 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-98bqc" event={"ID":"bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e","Type":"ContainerStarted","Data":"2a141e4f413d2d7c967c4c0ae98d54de695d780fbd5c00e8c3b4df64255f8b9c"} Jul 16 12:21:44.915806 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:44.915784 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hg7qx" event={"ID":"029311a2-84cd-4e64-81c5-65c0b73d9aad","Type":"ContainerStarted","Data":"2bd1f06a410009fe7a688061145446adc1c3a936e63c0f79d2bd41938b9e1df9"} Jul 16 12:21:44.931330 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:44.931262 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-98bqc" podStartSLOduration=67.917958319 podStartE2EDuration="1m8.931242643s" totalImagesPullingTime="1.013284324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:44.930768883 +0000 UTC m=+68.792670130" watchObservedRunningTime="2026-07-16 12:21:44.931242643 +0000 UTC m=+68.793143891" Jul 16 12:21:46.830131 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.830095 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-86446784b8-qlxst"] Jul 16 12:21:46.834259 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.834233 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:46.837329 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.837280 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 16 12:21:46.837586 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.837570 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 16 12:21:46.837664 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.837647 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-xcf88\"" Jul 16 12:21:46.837842 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.837572 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 16 12:21:46.838158 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.838131 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-97lgbfa057e7o\"" Jul 16 12:21:46.838503 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.838480 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 16 12:21:46.839510 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.838712 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 16 12:21:46.845110 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.845090 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-86446784b8-qlxst"] Jul 16 12:21:46.917437 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.917399 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrgkg\" (UniqueName: \"kubernetes.io/projected/0fb8710e-ff74-4545-b298-654f7a9752a2-kube-api-access-jrgkg\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:46.917596 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.917450 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-tls\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:46.917596 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.917495 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:46.917596 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.917525 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:46.917596 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.917557 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-grpc-tls\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:46.917596 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.917589 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:46.917842 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.917661 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:46.917842 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:46.917715 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fb8710e-ff74-4545-b298-654f7a9752a2-metrics-client-ca\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.018589 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.018554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrgkg\" (UniqueName: \"kubernetes.io/projected/0fb8710e-ff74-4545-b298-654f7a9752a2-kube-api-access-jrgkg\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.018589 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.018595 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-tls\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.018829 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.018626 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.018829 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.018654 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.018829 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.018677 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-grpc-tls\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.018829 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.018699 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.018829 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.018723 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.018829 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.018751 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fb8710e-ff74-4545-b298-654f7a9752a2-metrics-client-ca\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.019510 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.019489 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0fb8710e-ff74-4545-b298-654f7a9752a2-metrics-client-ca\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.022475 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.022447 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.022598 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.022572 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.023079 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.023056 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-tls\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.023164 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.023071 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.023216 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.023175 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.023415 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.023395 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0fb8710e-ff74-4545-b298-654f7a9752a2-secret-grpc-tls\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.028602 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.028577 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrgkg\" (UniqueName: \"kubernetes.io/projected/0fb8710e-ff74-4545-b298-654f7a9752a2-kube-api-access-jrgkg\") pod \"thanos-querier-86446784b8-qlxst\" (UID: \"0fb8710e-ff74-4545-b298-654f7a9752a2\") " pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:47.147044 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:47.147017 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:48.246913 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.246888 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-655886bb64-7s7t7"] Jul 16 12:21:48.253280 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.253233 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.256054 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.255965 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9sfjs\"" Jul 16 12:21:48.256214 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.256196 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4at4hjishqk1s\"" Jul 16 12:21:48.256559 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.256541 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 16 12:21:48.256648 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.256570 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 16 12:21:48.256648 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.256632 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 16 12:21:48.256785 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.256769 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 16 12:21:48.261321 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.261255 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-655886bb64-7s7t7"] Jul 16 12:21:48.329555 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.329466 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-secret-metrics-server-tls\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.329555 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.329513 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.329733 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.329602 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xcst\" (UniqueName: \"kubernetes.io/projected/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-kube-api-access-8xcst\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.329733 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.329644 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-metrics-server-audit-profiles\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.329733 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.329701 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-secret-metrics-server-client-certs\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.329873 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.329732 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-client-ca-bundle\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.329873 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.329772 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-audit-log\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.385251 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.385216 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-86446784b8-qlxst"] Jul 16 12:21:48.388788 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:21:48.388747 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fb8710e_ff74_4545_b298_654f7a9752a2.slice/crio-4e6e14c73ebef8c3e4725f44907284d34424ffde74d0dab4a1111c778a4388eb WatchSource:0}: Error finding container 4e6e14c73ebef8c3e4725f44907284d34424ffde74d0dab4a1111c778a4388eb: Status 404 returned error can't find the container with id 4e6e14c73ebef8c3e4725f44907284d34424ffde74d0dab4a1111c778a4388eb Jul 16 12:21:48.423731 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.423703 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp"] Jul 16 12:21:48.426773 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.426752 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84c689c7c7-mrzsd"] Jul 16 12:21:48.426884 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.426868 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" Jul 16 12:21:48.429112 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.429089 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 16 12:21:48.429323 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.429282 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-7b7m9\"" Jul 16 12:21:48.430348 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.430332 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.430592 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.430517 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-secret-metrics-server-tls\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.430592 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.430560 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.430744 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.430666 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xcst\" (UniqueName: \"kubernetes.io/projected/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-kube-api-access-8xcst\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.430744 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.430714 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-metrics-server-audit-profiles\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.430845 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.430747 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-secret-metrics-server-client-certs\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.430845 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.430784 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-client-ca-bundle\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.431229 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.431181 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.431447 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.431359 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-audit-log\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.431744 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.431721 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-audit-log\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.431814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.431772 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-metrics-server-audit-profiles\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.432629 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.432608 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 12:21:48.432892 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.432853 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-snwwc\"" Jul 16 12:21:48.432998 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.432908 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 12:21:48.432998 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.432936 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 12:21:48.433315 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.433278 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 12:21:48.433412 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.433331 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 12:21:48.433467 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.433409 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-secret-metrics-server-tls\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.433555 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.433539 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 12:21:48.433606 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.433555 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-secret-metrics-server-client-certs\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.433797 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.433603 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-client-ca-bundle\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.434354 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.434334 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 12:21:48.439566 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.439543 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp"] Jul 16 12:21:48.440713 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.440669 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84c689c7c7-mrzsd"] Jul 16 12:21:48.440915 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.440876 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 12:21:48.442820 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.442799 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xcst\" (UniqueName: \"kubernetes.io/projected/dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7-kube-api-access-8xcst\") pod \"metrics-server-655886bb64-7s7t7\" (UID: \"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7\") " pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.532605 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.532531 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-serving-cert\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.532605 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.532570 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-oauth-config\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.532788 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.532634 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-config\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.532788 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.532671 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtgvc\" (UniqueName: \"kubernetes.io/projected/a3c84aee-058d-4745-bf8e-cdb49a140ad1-kube-api-access-dtgvc\") pod \"monitoring-plugin-5f68df7f58-6k4gp\" (UID: \"a3c84aee-058d-4745-bf8e-cdb49a140ad1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" Jul 16 12:21:48.532788 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.532769 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-service-ca\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.533112 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.532810 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-oauth-serving-cert\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.533112 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.532858 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a3c84aee-058d-4745-bf8e-cdb49a140ad1-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-6k4gp\" (UID: \"a3c84aee-058d-4745-bf8e-cdb49a140ad1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" Jul 16 12:21:48.533112 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.532890 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-trusted-ca-bundle\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.533112 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.532921 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kpsg\" (UniqueName: \"kubernetes.io/projected/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-kube-api-access-7kpsg\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.597943 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.597914 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:21:48.633748 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.633719 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-trusted-ca-bundle\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.633863 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.633757 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kpsg\" (UniqueName: \"kubernetes.io/projected/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-kube-api-access-7kpsg\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.633863 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.633837 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-serving-cert\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.633949 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.633872 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-oauth-config\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.633949 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.633905 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-config\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.633949 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.633937 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtgvc\" (UniqueName: \"kubernetes.io/projected/a3c84aee-058d-4745-bf8e-cdb49a140ad1-kube-api-access-dtgvc\") pod \"monitoring-plugin-5f68df7f58-6k4gp\" (UID: \"a3c84aee-058d-4745-bf8e-cdb49a140ad1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" Jul 16 12:21:48.634098 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.633973 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-service-ca\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.634098 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.634003 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-oauth-serving-cert\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.634098 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.634027 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a3c84aee-058d-4745-bf8e-cdb49a140ad1-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-6k4gp\" (UID: \"a3c84aee-058d-4745-bf8e-cdb49a140ad1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" Jul 16 12:21:48.634246 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:48.634125 2577 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 16 12:21:48.634246 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:21:48.634184 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c84aee-058d-4745-bf8e-cdb49a140ad1-monitoring-plugin-cert podName:a3c84aee-058d-4745-bf8e-cdb49a140ad1 nodeName:}" failed. No retries permitted until 2026-07-16 12:21:49.134164371 +0000 UTC m=+72.996065608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/a3c84aee-058d-4745-bf8e-cdb49a140ad1-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-6k4gp" (UID: "a3c84aee-058d-4745-bf8e-cdb49a140ad1") : secret "monitoring-plugin-cert" not found Jul 16 12:21:48.634980 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.634895 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-config\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.634980 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.634928 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-oauth-serving-cert\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.634980 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.634977 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-service-ca\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.635169 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.635072 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-trusted-ca-bundle\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.636860 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.636836 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-serving-cert\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.637124 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.637102 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-oauth-config\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.643212 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.643193 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kpsg\" (UniqueName: \"kubernetes.io/projected/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-kube-api-access-7kpsg\") pod \"console-84c689c7c7-mrzsd\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.643318 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.643255 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtgvc\" (UniqueName: \"kubernetes.io/projected/a3c84aee-058d-4745-bf8e-cdb49a140ad1-kube-api-access-dtgvc\") pod \"monitoring-plugin-5f68df7f58-6k4gp\" (UID: \"a3c84aee-058d-4745-bf8e-cdb49a140ad1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" Jul 16 12:21:48.730419 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.730391 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-655886bb64-7s7t7"] Jul 16 12:21:48.733053 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:21:48.733026 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc78434d_b99a_44d8_9d74_4d1b3e3ba1f7.slice/crio-f89cfe062c692b5bf78372bfb628f9f0bfcf2396de479aeb9f147fb6642e584b WatchSource:0}: Error finding container f89cfe062c692b5bf78372bfb628f9f0bfcf2396de479aeb9f147fb6642e584b: Status 404 returned error can't find the container with id f89cfe062c692b5bf78372bfb628f9f0bfcf2396de479aeb9f147fb6642e584b Jul 16 12:21:48.771381 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.771356 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:48.907412 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.907365 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84c689c7c7-mrzsd"] Jul 16 12:21:48.910606 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:21:48.910566 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod745d3a66_9d89_4f6b_b8ed_0ef4e7558bf7.slice/crio-43820520b7db65e262ab724fe982aa13a23bb36ac347bec3a2f8982006337e9e WatchSource:0}: Error finding container 43820520b7db65e262ab724fe982aa13a23bb36ac347bec3a2f8982006337e9e: Status 404 returned error can't find the container with id 43820520b7db65e262ab724fe982aa13a23bb36ac347bec3a2f8982006337e9e Jul 16 12:21:48.932151 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.932122 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" event={"ID":"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7","Type":"ContainerStarted","Data":"f89cfe062c692b5bf78372bfb628f9f0bfcf2396de479aeb9f147fb6642e584b"} Jul 16 12:21:48.933858 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.933830 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4qrhp" event={"ID":"ac6ef832-3b6b-4b74-a5bf-3823389a6d1c","Type":"ContainerStarted","Data":"d79bcda18a8c4c72d2208f0069173bf2df53b854ad87369ad625fa9a0463798d"} Jul 16 12:21:48.933979 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.933956 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:21:48.935593 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.935574 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wjdcb" event={"ID":"4a3e09d3-661a-4a7f-8741-f39758ea7303","Type":"ContainerStarted","Data":"414d6b4172e0884f5aced141bb44b6fcdd2f72f2258b205e45c247706308fa9b"} Jul 16 12:21:48.936811 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.936789 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" event={"ID":"0fb8710e-ff74-4545-b298-654f7a9752a2","Type":"ContainerStarted","Data":"4e6e14c73ebef8c3e4725f44907284d34424ffde74d0dab4a1111c778a4388eb"} Jul 16 12:21:48.938339 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.938312 2577 generic.go:356] "Generic (PLEG): container finished" podID="029311a2-84cd-4e64-81c5-65c0b73d9aad" containerID="197fa89b4e7a9ed5f7d4b2a6e799f8c838e0053a571a7ca5d300929fe3e22187" exitCode=0 Jul 16 12:21:48.938418 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.938379 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hg7qx" event={"ID":"029311a2-84cd-4e64-81c5-65c0b73d9aad","Type":"ContainerDied","Data":"197fa89b4e7a9ed5f7d4b2a6e799f8c838e0053a571a7ca5d300929fe3e22187"} Jul 16 12:21:48.939659 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.939631 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c689c7c7-mrzsd" event={"ID":"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7","Type":"ContainerStarted","Data":"43820520b7db65e262ab724fe982aa13a23bb36ac347bec3a2f8982006337e9e"} Jul 16 12:21:48.977398 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.977356 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wjdcb" podStartSLOduration=66.802736898 podStartE2EDuration="1m11.977340548s" totalImagesPullingTime="5.17460365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 12:20:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:48.976612955 +0000 UTC m=+72.838514202" watchObservedRunningTime="2026-07-16 12:21:48.977340548 +0000 UTC m=+72.839241771" Jul 16 12:21:48.977716 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:48.977689 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4qrhp" podStartSLOduration=67.819372185 podStartE2EDuration="1m12.977684037s" totalImagesPullingTime="5.158311852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:48.960561733 +0000 UTC m=+72.822462979" watchObservedRunningTime="2026-07-16 12:21:48.977684037 +0000 UTC m=+72.839585261" Jul 16 12:21:49.138197 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:49.138170 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a3c84aee-058d-4745-bf8e-cdb49a140ad1-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-6k4gp\" (UID: \"a3c84aee-058d-4745-bf8e-cdb49a140ad1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" Jul 16 12:21:49.141012 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:49.140988 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a3c84aee-058d-4745-bf8e-cdb49a140ad1-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-6k4gp\" (UID: \"a3c84aee-058d-4745-bf8e-cdb49a140ad1\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" Jul 16 12:21:49.364560 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:49.364484 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" Jul 16 12:21:49.520132 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:49.520098 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp"] Jul 16 12:21:49.524797 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:21:49.523278 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c84aee_058d_4745_bf8e_cdb49a140ad1.slice/crio-35384a9df72600a6b0c7118f35eb42bdc0949304e8d568a031b0876169bd02d7 WatchSource:0}: Error finding container 35384a9df72600a6b0c7118f35eb42bdc0949304e8d568a031b0876169bd02d7: Status 404 returned error can't find the container with id 35384a9df72600a6b0c7118f35eb42bdc0949304e8d568a031b0876169bd02d7 Jul 16 12:21:49.951283 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:49.951247 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hg7qx" event={"ID":"029311a2-84cd-4e64-81c5-65c0b73d9aad","Type":"ContainerStarted","Data":"4df889af50ab5597c63f62ace54902c36335e6cf0f4892ba1105948ab23308ba"} Jul 16 12:21:49.951530 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:49.951341 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hg7qx" event={"ID":"029311a2-84cd-4e64-81c5-65c0b73d9aad","Type":"ContainerStarted","Data":"c3f871a545077ebfda8a9f2169e63e3d30d85a6ea5769d47e0ccd13c989cfbb7"} Jul 16 12:21:49.953558 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:49.953132 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" event={"ID":"a3c84aee-058d-4745-bf8e-cdb49a140ad1","Type":"ContainerStarted","Data":"35384a9df72600a6b0c7118f35eb42bdc0949304e8d568a031b0876169bd02d7"} Jul 16 12:21:49.975040 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:49.974983 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hg7qx" podStartSLOduration=2.739855932 podStartE2EDuration="6.974966018s" totalImagesPullingTime="4.152387541s" totalInitContainerRuntime="82.722545ms" isStatefulPod=true podCreationTimestamp="2026-07-16 12:21:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:49.973697466 +0000 UTC m=+73.835598723" watchObservedRunningTime="2026-07-16 12:21:49.974966018 +0000 UTC m=+73.836867266" Jul 16 12:21:50.857677 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:50.857646 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75f6d5c9cd-ssrzn" Jul 16 12:21:50.963132 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:50.963101 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" event={"ID":"a3c84aee-058d-4745-bf8e-cdb49a140ad1","Type":"ContainerStarted","Data":"099c36ad2858c154fa4939044db95ca2feb981fcd6b1c2ac6f92342deec6c6d5"} Jul 16 12:21:50.984373 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:50.984261 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" podStartSLOduration=1.630789296 podStartE2EDuration="2.984230347s" totalImagesPullingTime="1.353441051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:21:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:50.984132748 +0000 UTC m=+74.846033995" watchObservedRunningTime="2026-07-16 12:21:50.984230347 +0000 UTC m=+74.846131593" Jul 16 12:21:51.969896 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:51.969853 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" event={"ID":"0fb8710e-ff74-4545-b298-654f7a9752a2","Type":"ContainerStarted","Data":"42db4c88e693a1b7072af4f9e3f2f60ef92a3fa927d4494b942b40d0605f0555"} Jul 16 12:21:51.970353 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:51.969926 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" event={"ID":"0fb8710e-ff74-4545-b298-654f7a9752a2","Type":"ContainerStarted","Data":"28789c93101a15bd151a5708d2d95059338e96504582984a14e82ce352e5c88e"} Jul 16 12:21:51.970353 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:51.969967 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" event={"ID":"0fb8710e-ff74-4545-b298-654f7a9752a2","Type":"ContainerStarted","Data":"17a70f95ebdaed45eb7843809c6c49a2094261f3a47926ee5484ac32da03ab7b"} Jul 16 12:21:51.970353 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:51.970121 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" Jul 16 12:21:51.975794 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:51.975773 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-6k4gp" Jul 16 12:21:53.501723 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.501686 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84c689c7c7-mrzsd"] Jul 16 12:21:53.533360 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.533326 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5796c84848-8dkgd"] Jul 16 12:21:53.536489 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.536446 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.554773 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.554749 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5796c84848-8dkgd"] Jul 16 12:21:53.677081 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.677044 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/485a2f4a-15f3-495c-af00-0c383bd4a482-console-oauth-config\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.677259 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.677090 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-service-ca\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.677259 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.677107 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-trusted-ca-bundle\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.677259 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.677130 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-oauth-serving-cert\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.677259 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.677201 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f9cc\" (UniqueName: \"kubernetes.io/projected/485a2f4a-15f3-495c-af00-0c383bd4a482-kube-api-access-5f9cc\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.677444 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.677261 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/485a2f4a-15f3-495c-af00-0c383bd4a482-console-serving-cert\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.677444 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.677286 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-console-config\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.777693 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.777659 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/485a2f4a-15f3-495c-af00-0c383bd4a482-console-serving-cert\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.777693 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.777693 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-console-config\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.777933 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.777711 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/485a2f4a-15f3-495c-af00-0c383bd4a482-console-oauth-config\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.777933 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.777731 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-service-ca\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.778057 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.778028 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-trusted-ca-bundle\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.778134 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.778115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-oauth-serving-cert\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.778219 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.778190 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f9cc\" (UniqueName: \"kubernetes.io/projected/485a2f4a-15f3-495c-af00-0c383bd4a482-kube-api-access-5f9cc\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.778530 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.778510 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-console-config\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.779030 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.779005 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-oauth-serving-cert\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.779095 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.779072 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-trusted-ca-bundle\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.779366 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.779348 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-service-ca\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.780387 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.780367 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/485a2f4a-15f3-495c-af00-0c383bd4a482-console-serving-cert\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.780477 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.780432 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/485a2f4a-15f3-495c-af00-0c383bd4a482-console-oauth-config\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.791796 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.791747 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f9cc\" (UniqueName: \"kubernetes.io/projected/485a2f4a-15f3-495c-af00-0c383bd4a482-kube-api-access-5f9cc\") pod \"console-5796c84848-8dkgd\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.844795 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.844769 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:21:53.979132 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.979097 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c689c7c7-mrzsd" event={"ID":"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7","Type":"ContainerStarted","Data":"98eb3857a7e19aad53f8e81fea38afe5258a209d4084d5ce60a5464229f9bc4d"} Jul 16 12:21:53.980789 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.980759 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" event={"ID":"dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7","Type":"ContainerStarted","Data":"964e81f88c7ba087cfe68b8e4dc2c6ebeac3d140f02521b6388b822ef3e94368"} Jul 16 12:21:53.983501 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.983477 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" event={"ID":"0fb8710e-ff74-4545-b298-654f7a9752a2","Type":"ContainerStarted","Data":"b58792c90cbb64743065dcce0f3b405c0bfc6cd6c5516d01a0eb7286947decd5"} Jul 16 12:21:53.983594 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.983540 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" event={"ID":"0fb8710e-ff74-4545-b298-654f7a9752a2","Type":"ContainerStarted","Data":"27e843e1e386c345c32677f05fe0ddeb1ef5dc7b9f898e3711268ae57aab46bd"} Jul 16 12:21:53.983594 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.983590 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" event={"ID":"0fb8710e-ff74-4545-b298-654f7a9752a2","Type":"ContainerStarted","Data":"231d41e9364757d2ff51b64204a5606c5393cbe139d77c90e43a18589021281d"} Jul 16 12:21:53.983671 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.983661 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:21:53.998643 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:53.998627 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5796c84848-8dkgd"] Jul 16 12:21:54.000155 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:54.000109 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84c689c7c7-mrzsd" podStartSLOduration=1.7823127429999999 podStartE2EDuration="6.00009213s" totalImagesPullingTime="4.217779387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:21:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:53.999606858 +0000 UTC m=+77.861508104" watchObservedRunningTime="2026-07-16 12:21:54.00009213 +0000 UTC m=+77.861993389" Jul 16 12:21:54.001655 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:21:54.001630 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod485a2f4a_15f3_495c_af00_0c383bd4a482.slice/crio-d4181255617328eb1264dd26ab127d0b8e851288cadd07cc1f1420e05e8eb294 WatchSource:0}: Error finding container d4181255617328eb1264dd26ab127d0b8e851288cadd07cc1f1420e05e8eb294: Status 404 returned error can't find the container with id d4181255617328eb1264dd26ab127d0b8e851288cadd07cc1f1420e05e8eb294 Jul 16 12:21:54.020644 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:54.020608 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" podStartSLOduration=1.632470417 podStartE2EDuration="6.020595881s" totalImagesPullingTime="4.388125464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:21:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:54.020144256 +0000 UTC m=+77.882045502" watchObservedRunningTime="2026-07-16 12:21:54.020595881 +0000 UTC m=+77.882497104" Jul 16 12:21:54.042782 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:54.042718 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" podStartSLOduration=3.513898346 podStartE2EDuration="8.042703731s" totalImagesPullingTime="4.528805385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:21:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:54.041399463 +0000 UTC m=+77.903300754" watchObservedRunningTime="2026-07-16 12:21:54.042703731 +0000 UTC m=+77.904604975" Jul 16 12:21:54.988188 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:54.988141 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5796c84848-8dkgd" event={"ID":"485a2f4a-15f3-495c-af00-0c383bd4a482","Type":"ContainerStarted","Data":"07bb3b5e2db2b40699da2e29f46fa7bcd693c9d61b718bfc1ec9c6d8da7877b0"} Jul 16 12:21:54.988589 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:54.988207 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5796c84848-8dkgd" event={"ID":"485a2f4a-15f3-495c-af00-0c383bd4a482","Type":"ContainerStarted","Data":"d4181255617328eb1264dd26ab127d0b8e851288cadd07cc1f1420e05e8eb294"} Jul 16 12:21:55.009634 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:55.009576 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5796c84848-8dkgd" podStartSLOduration=2.00956253 podStartE2EDuration="2.00956253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:21:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:21:55.008482216 +0000 UTC m=+78.870383485" watchObservedRunningTime="2026-07-16 12:21:55.00956253 +0000 UTC m=+78.871463775" Jul 16 12:21:58.772269 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:58.772155 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:21:59.994869 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:21:59.994841 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-86446784b8-qlxst" Jul 16 12:22:03.844935 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:03.844890 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:22:03.845419 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:03.844965 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:22:03.849644 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:03.849621 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:22:04.021948 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:04.021920 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:22:08.598134 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:08.598101 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:22:08.598587 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:08.598202 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:22:19.956060 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:19.956030 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4qrhp" Jul 16 12:22:20.008437 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.008404 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84c689c7c7-mrzsd" podUID="745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7" containerName="console" containerID="cri-o://98eb3857a7e19aad53f8e81fea38afe5258a209d4084d5ce60a5464229f9bc4d" gracePeriod=15 Jul 16 12:22:20.790204 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.790181 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:22:20.890523 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.890496 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kpsg\" (UniqueName: \"kubernetes.io/projected/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-kube-api-access-7kpsg\") pod \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " Jul 16 12:22:20.890523 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.890527 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-config\") pod \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " Jul 16 12:22:20.890725 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.890572 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-oauth-config\") pod \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " Jul 16 12:22:20.890725 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.890595 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-trusted-ca-bundle\") pod \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " Jul 16 12:22:20.890725 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.890612 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-oauth-serving-cert\") pod \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " Jul 16 12:22:20.890725 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.890643 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-service-ca\") pod \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " Jul 16 12:22:20.890725 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.890669 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-serving-cert\") pod \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\" (UID: \"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7\") " Jul 16 12:22:20.891028 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.890999 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-config" (OuterVolumeSpecName: "console-config") pod "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7" (UID: "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:22:20.891121 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.891093 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7" (UID: "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:22:20.891204 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.891127 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7" (UID: "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:22:20.891204 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.891132 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-service-ca" (OuterVolumeSpecName: "service-ca") pod "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7" (UID: "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:22:20.892890 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.892870 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-kube-api-access-7kpsg" (OuterVolumeSpecName: "kube-api-access-7kpsg") pod "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7" (UID: "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7"). InnerVolumeSpecName "kube-api-access-7kpsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:22:20.893435 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.893416 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7" (UID: "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:22:20.893501 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.893450 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7" (UID: "745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:22:20.991908 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.991881 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kpsg\" (UniqueName: \"kubernetes.io/projected/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-kube-api-access-7kpsg\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:22:20.991908 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.991904 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-config\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:22:20.992236 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.991914 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-oauth-config\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:22:20.992236 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.991923 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-trusted-ca-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:22:20.992236 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.991931 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-oauth-serving-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:22:20.992236 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.991958 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-service-ca\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:22:20.992236 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:20.991966 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7-console-serving-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:22:21.079968 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:21.079907 2577 generic.go:356] "Generic (PLEG): container finished" podID="745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7" containerID="98eb3857a7e19aad53f8e81fea38afe5258a209d4084d5ce60a5464229f9bc4d" exitCode=0 Jul 16 12:22:21.079968 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:21.079949 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84c689c7c7-mrzsd" Jul 16 12:22:21.079968 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:21.079966 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c689c7c7-mrzsd" event={"ID":"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7","Type":"ContainerDied","Data":"98eb3857a7e19aad53f8e81fea38afe5258a209d4084d5ce60a5464229f9bc4d"} Jul 16 12:22:21.080151 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:21.080005 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84c689c7c7-mrzsd" event={"ID":"745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7","Type":"ContainerDied","Data":"43820520b7db65e262ab724fe982aa13a23bb36ac347bec3a2f8982006337e9e"} Jul 16 12:22:21.080151 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:21.080023 2577 scope.go:117] "RemoveContainer" containerID="98eb3857a7e19aad53f8e81fea38afe5258a209d4084d5ce60a5464229f9bc4d" Jul 16 12:22:21.089740 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:21.089715 2577 scope.go:117] "RemoveContainer" containerID="98eb3857a7e19aad53f8e81fea38afe5258a209d4084d5ce60a5464229f9bc4d" Jul 16 12:22:21.090285 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:22:21.090257 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98eb3857a7e19aad53f8e81fea38afe5258a209d4084d5ce60a5464229f9bc4d\": container with ID starting with 98eb3857a7e19aad53f8e81fea38afe5258a209d4084d5ce60a5464229f9bc4d not found: ID does not exist" containerID="98eb3857a7e19aad53f8e81fea38afe5258a209d4084d5ce60a5464229f9bc4d" Jul 16 12:22:21.090410 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:21.090310 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98eb3857a7e19aad53f8e81fea38afe5258a209d4084d5ce60a5464229f9bc4d"} err="failed to get container status \"98eb3857a7e19aad53f8e81fea38afe5258a209d4084d5ce60a5464229f9bc4d\": rpc error: code = NotFound desc = could not find container \"98eb3857a7e19aad53f8e81fea38afe5258a209d4084d5ce60a5464229f9bc4d\": container with ID starting with 98eb3857a7e19aad53f8e81fea38afe5258a209d4084d5ce60a5464229f9bc4d not found: ID does not exist" Jul 16 12:22:21.107090 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:21.107063 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84c689c7c7-mrzsd"] Jul 16 12:22:21.111162 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:21.111143 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84c689c7c7-mrzsd"] Jul 16 12:22:22.638543 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:22.638509 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7" path="/var/lib/kubelet/pods/745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7/volumes" Jul 16 12:22:28.603395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:28.603369 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:22:28.607037 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:22:28.607018 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-655886bb64-7s7t7" Jul 16 12:23:12.774787 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.774747 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9bdd6df8d-6m4gf"] Jul 16 12:23:12.775388 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.775098 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7" containerName="console" Jul 16 12:23:12.775388 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.775116 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7" containerName="console" Jul 16 12:23:12.775388 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.775192 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="745d3a66-9d89-4f6b-b8ed-0ef4e7558bf7" containerName="console" Jul 16 12:23:12.778132 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.778112 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.789830 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.789805 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9bdd6df8d-6m4gf"] Jul 16 12:23:12.864756 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.864730 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-oauth-serving-cert\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.864882 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.864766 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-oauth-config\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.864882 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.864785 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-serving-cert\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.864882 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.864800 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-trusted-ca-bundle\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.864882 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.864823 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-service-ca\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.864882 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.864851 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-config\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.864882 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.864872 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7stw\" (UniqueName: \"kubernetes.io/projected/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-kube-api-access-b7stw\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.966027 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.965983 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-oauth-serving-cert\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.966027 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.966031 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-oauth-config\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.966265 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.966048 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-serving-cert\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.966265 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.966065 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-trusted-ca-bundle\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.966265 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.966081 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-service-ca\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.966265 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.966225 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-config\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.966523 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.966274 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7stw\" (UniqueName: \"kubernetes.io/projected/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-kube-api-access-b7stw\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.966891 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.966842 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-oauth-serving-cert\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.966891 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.966863 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-service-ca\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.967006 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.966971 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-config\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.967271 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.967250 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-trusted-ca-bundle\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.968764 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.968735 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-oauth-config\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.968901 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.968881 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-serving-cert\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:12.975783 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:12.975763 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7stw\" (UniqueName: \"kubernetes.io/projected/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-kube-api-access-b7stw\") pod \"console-9bdd6df8d-6m4gf\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:13.086529 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:13.086443 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:13.215631 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:13.215579 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9bdd6df8d-6m4gf"] Jul 16 12:23:13.218379 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:23:13.218353 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353ebb58_92bb_4fa3_904c_3c8ab81b2e72.slice/crio-1c02076f3e73043e18648f8445a600ebb0bb3365ffce2c8ce6a43fc6a71e64de WatchSource:0}: Error finding container 1c02076f3e73043e18648f8445a600ebb0bb3365ffce2c8ce6a43fc6a71e64de: Status 404 returned error can't find the container with id 1c02076f3e73043e18648f8445a600ebb0bb3365ffce2c8ce6a43fc6a71e64de Jul 16 12:23:13.256664 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:13.256632 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9bdd6df8d-6m4gf" event={"ID":"353ebb58-92bb-4fa3-904c-3c8ab81b2e72","Type":"ContainerStarted","Data":"1c02076f3e73043e18648f8445a600ebb0bb3365ffce2c8ce6a43fc6a71e64de"} Jul 16 12:23:14.263373 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:14.263340 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9bdd6df8d-6m4gf" event={"ID":"353ebb58-92bb-4fa3-904c-3c8ab81b2e72","Type":"ContainerStarted","Data":"ed1c0f34f234e8af96dc93da508c9050d5400e77a2f0dd050f7129792d92bf45"} Jul 16 12:23:14.303261 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:14.303214 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9bdd6df8d-6m4gf" podStartSLOduration=2.303199545 podStartE2EDuration="2.303199545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:23:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:23:14.303178635 +0000 UTC m=+158.165079881" watchObservedRunningTime="2026-07-16 12:23:14.303199545 +0000 UTC m=+158.165100790" Jul 16 12:23:23.087185 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:23.087148 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:23.087609 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:23.087230 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:23.092239 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:23.092212 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:23.297204 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:23.297178 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:23:23.346656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:23.346576 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5796c84848-8dkgd"] Jul 16 12:23:48.367560 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.367494 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5796c84848-8dkgd" podUID="485a2f4a-15f3-495c-af00-0c383bd4a482" containerName="console" containerID="cri-o://07bb3b5e2db2b40699da2e29f46fa7bcd693c9d61b718bfc1ec9c6d8da7877b0" gracePeriod=15 Jul 16 12:23:48.608839 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.608813 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:23:48.635641 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.635560 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/485a2f4a-15f3-495c-af00-0c383bd4a482-console-serving-cert\") pod \"485a2f4a-15f3-495c-af00-0c383bd4a482\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " Jul 16 12:23:48.635641 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.635607 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/485a2f4a-15f3-495c-af00-0c383bd4a482-console-oauth-config\") pod \"485a2f4a-15f3-495c-af00-0c383bd4a482\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " Jul 16 12:23:48.635854 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.635643 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-trusted-ca-bundle\") pod \"485a2f4a-15f3-495c-af00-0c383bd4a482\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " Jul 16 12:23:48.635854 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.635670 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5f9cc\" (UniqueName: \"kubernetes.io/projected/485a2f4a-15f3-495c-af00-0c383bd4a482-kube-api-access-5f9cc\") pod \"485a2f4a-15f3-495c-af00-0c383bd4a482\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " Jul 16 12:23:48.635854 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.635694 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-console-config\") pod \"485a2f4a-15f3-495c-af00-0c383bd4a482\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " Jul 16 12:23:48.635854 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.635751 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-service-ca\") pod \"485a2f4a-15f3-495c-af00-0c383bd4a482\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " Jul 16 12:23:48.635854 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.635771 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-oauth-serving-cert\") pod \"485a2f4a-15f3-495c-af00-0c383bd4a482\" (UID: \"485a2f4a-15f3-495c-af00-0c383bd4a482\") " Jul 16 12:23:48.636515 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.636231 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "485a2f4a-15f3-495c-af00-0c383bd4a482" (UID: "485a2f4a-15f3-495c-af00-0c383bd4a482"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:23:48.636678 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.636502 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-console-config" (OuterVolumeSpecName: "console-config") pod "485a2f4a-15f3-495c-af00-0c383bd4a482" (UID: "485a2f4a-15f3-495c-af00-0c383bd4a482"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:23:48.636773 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.636675 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "485a2f4a-15f3-495c-af00-0c383bd4a482" (UID: "485a2f4a-15f3-495c-af00-0c383bd4a482"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:23:48.637031 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.636992 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-service-ca" (OuterVolumeSpecName: "service-ca") pod "485a2f4a-15f3-495c-af00-0c383bd4a482" (UID: "485a2f4a-15f3-495c-af00-0c383bd4a482"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:23:48.638151 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.638117 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/485a2f4a-15f3-495c-af00-0c383bd4a482-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "485a2f4a-15f3-495c-af00-0c383bd4a482" (UID: "485a2f4a-15f3-495c-af00-0c383bd4a482"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:23:48.638253 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.638234 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/485a2f4a-15f3-495c-af00-0c383bd4a482-kube-api-access-5f9cc" (OuterVolumeSpecName: "kube-api-access-5f9cc") pod "485a2f4a-15f3-495c-af00-0c383bd4a482" (UID: "485a2f4a-15f3-495c-af00-0c383bd4a482"). InnerVolumeSpecName "kube-api-access-5f9cc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:23:48.638367 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.638346 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/485a2f4a-15f3-495c-af00-0c383bd4a482-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "485a2f4a-15f3-495c-af00-0c383bd4a482" (UID: "485a2f4a-15f3-495c-af00-0c383bd4a482"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:23:48.736851 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.736824 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-service-ca\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:23:48.736851 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.736849 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-oauth-serving-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:23:48.736851 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.736858 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/485a2f4a-15f3-495c-af00-0c383bd4a482-console-serving-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:23:48.737104 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.736866 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/485a2f4a-15f3-495c-af00-0c383bd4a482-console-oauth-config\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:23:48.737104 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.736874 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-trusted-ca-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:23:48.737104 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.736882 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5f9cc\" (UniqueName: \"kubernetes.io/projected/485a2f4a-15f3-495c-af00-0c383bd4a482-kube-api-access-5f9cc\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:23:48.737104 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:48.736891 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/485a2f4a-15f3-495c-af00-0c383bd4a482-console-config\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:23:49.380607 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:49.380575 2577 generic.go:356] "Generic (PLEG): container finished" podID="485a2f4a-15f3-495c-af00-0c383bd4a482" containerID="07bb3b5e2db2b40699da2e29f46fa7bcd693c9d61b718bfc1ec9c6d8da7877b0" exitCode=0 Jul 16 12:23:49.381077 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:49.380622 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5796c84848-8dkgd" Jul 16 12:23:49.381077 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:49.380669 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5796c84848-8dkgd" event={"ID":"485a2f4a-15f3-495c-af00-0c383bd4a482","Type":"ContainerDied","Data":"07bb3b5e2db2b40699da2e29f46fa7bcd693c9d61b718bfc1ec9c6d8da7877b0"} Jul 16 12:23:49.381077 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:49.380727 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5796c84848-8dkgd" event={"ID":"485a2f4a-15f3-495c-af00-0c383bd4a482","Type":"ContainerDied","Data":"d4181255617328eb1264dd26ab127d0b8e851288cadd07cc1f1420e05e8eb294"} Jul 16 12:23:49.381077 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:49.380760 2577 scope.go:117] "RemoveContainer" containerID="07bb3b5e2db2b40699da2e29f46fa7bcd693c9d61b718bfc1ec9c6d8da7877b0" Jul 16 12:23:49.389105 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:49.389090 2577 scope.go:117] "RemoveContainer" containerID="07bb3b5e2db2b40699da2e29f46fa7bcd693c9d61b718bfc1ec9c6d8da7877b0" Jul 16 12:23:49.389383 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:23:49.389365 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07bb3b5e2db2b40699da2e29f46fa7bcd693c9d61b718bfc1ec9c6d8da7877b0\": container with ID starting with 07bb3b5e2db2b40699da2e29f46fa7bcd693c9d61b718bfc1ec9c6d8da7877b0 not found: ID does not exist" containerID="07bb3b5e2db2b40699da2e29f46fa7bcd693c9d61b718bfc1ec9c6d8da7877b0" Jul 16 12:23:49.389448 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:49.389393 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07bb3b5e2db2b40699da2e29f46fa7bcd693c9d61b718bfc1ec9c6d8da7877b0"} err="failed to get container status \"07bb3b5e2db2b40699da2e29f46fa7bcd693c9d61b718bfc1ec9c6d8da7877b0\": rpc error: code = NotFound desc = could not find container \"07bb3b5e2db2b40699da2e29f46fa7bcd693c9d61b718bfc1ec9c6d8da7877b0\": container with ID starting with 07bb3b5e2db2b40699da2e29f46fa7bcd693c9d61b718bfc1ec9c6d8da7877b0 not found: ID does not exist" Jul 16 12:23:49.402007 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:49.401982 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5796c84848-8dkgd"] Jul 16 12:23:49.405513 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:49.405494 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5796c84848-8dkgd"] Jul 16 12:23:50.637609 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:23:50.637579 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="485a2f4a-15f3-495c-af00-0c383bd4a482" path="/var/lib/kubelet/pods/485a2f4a-15f3-495c-af00-0c383bd4a482/volumes" Jul 16 12:24:18.341589 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.341558 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq"] Jul 16 12:24:18.341987 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.341833 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="485a2f4a-15f3-495c-af00-0c383bd4a482" containerName="console" Jul 16 12:24:18.341987 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.341843 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="485a2f4a-15f3-495c-af00-0c383bd4a482" containerName="console" Jul 16 12:24:18.341987 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.341887 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="485a2f4a-15f3-495c-af00-0c383bd4a482" containerName="console" Jul 16 12:24:18.344673 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.344655 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" Jul 16 12:24:18.347383 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.347355 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 12:24:18.347504 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.347441 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 12:24:18.348348 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.348328 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-578t6\"" Jul 16 12:24:18.359677 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.359652 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq"] Jul 16 12:24:18.448440 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.448410 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3abe92f-607b-4882-8e5f-5db96bf25b8f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq\" (UID: \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" Jul 16 12:24:18.448617 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.448453 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3abe92f-607b-4882-8e5f-5db96bf25b8f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq\" (UID: \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" Jul 16 12:24:18.448617 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.448477 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvjjm\" (UniqueName: \"kubernetes.io/projected/b3abe92f-607b-4882-8e5f-5db96bf25b8f-kube-api-access-fvjjm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq\" (UID: \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" Jul 16 12:24:18.548974 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.548946 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3abe92f-607b-4882-8e5f-5db96bf25b8f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq\" (UID: \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" Jul 16 12:24:18.549071 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.548982 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvjjm\" (UniqueName: \"kubernetes.io/projected/b3abe92f-607b-4882-8e5f-5db96bf25b8f-kube-api-access-fvjjm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq\" (UID: \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" Jul 16 12:24:18.549071 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.549032 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3abe92f-607b-4882-8e5f-5db96bf25b8f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq\" (UID: \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" Jul 16 12:24:18.549330 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.549284 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3abe92f-607b-4882-8e5f-5db96bf25b8f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq\" (UID: \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" Jul 16 12:24:18.549378 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.549355 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3abe92f-607b-4882-8e5f-5db96bf25b8f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq\" (UID: \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" Jul 16 12:24:18.558975 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.558947 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvjjm\" (UniqueName: \"kubernetes.io/projected/b3abe92f-607b-4882-8e5f-5db96bf25b8f-kube-api-access-fvjjm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq\" (UID: \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" Jul 16 12:24:18.654088 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.654067 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" Jul 16 12:24:18.781386 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:18.781358 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq"] Jul 16 12:24:18.784154 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:24:18.784125 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3abe92f_607b_4882_8e5f_5db96bf25b8f.slice/crio-53edad9ebe1aefc61c09e738bc49b4ce881efb3d110a97f086fc1c13c09f1c6d WatchSource:0}: Error finding container 53edad9ebe1aefc61c09e738bc49b4ce881efb3d110a97f086fc1c13c09f1c6d: Status 404 returned error can't find the container with id 53edad9ebe1aefc61c09e738bc49b4ce881efb3d110a97f086fc1c13c09f1c6d Jul 16 12:24:19.484995 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:19.484960 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" event={"ID":"b3abe92f-607b-4882-8e5f-5db96bf25b8f","Type":"ContainerStarted","Data":"53edad9ebe1aefc61c09e738bc49b4ce881efb3d110a97f086fc1c13c09f1c6d"} Jul 16 12:24:25.509358 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:25.509254 2577 generic.go:356] "Generic (PLEG): container finished" podID="b3abe92f-607b-4882-8e5f-5db96bf25b8f" containerID="0d6666b2d265c5847e867c54d5403565ed4b65ea9cd6441983eafed95585b153" exitCode=0 Jul 16 12:24:25.509701 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:25.509348 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" event={"ID":"b3abe92f-607b-4882-8e5f-5db96bf25b8f","Type":"ContainerDied","Data":"0d6666b2d265c5847e867c54d5403565ed4b65ea9cd6441983eafed95585b153"} Jul 16 12:24:27.518735 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:27.518648 2577 generic.go:356] "Generic (PLEG): container finished" podID="b3abe92f-607b-4882-8e5f-5db96bf25b8f" containerID="fa509fb19fbda7ec0c30bc0b620d8549ae9da2137b228c33644e45b170fe94fb" exitCode=0 Jul 16 12:24:27.518735 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:27.518724 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" event={"ID":"b3abe92f-607b-4882-8e5f-5db96bf25b8f","Type":"ContainerDied","Data":"fa509fb19fbda7ec0c30bc0b620d8549ae9da2137b228c33644e45b170fe94fb"} Jul 16 12:24:34.549175 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:34.549139 2577 generic.go:356] "Generic (PLEG): container finished" podID="b3abe92f-607b-4882-8e5f-5db96bf25b8f" containerID="2063d4f1b06c8532af34fbe57edc75cd5fa2b1cda1e41d43bf06b5fcf4a4a1da" exitCode=0 Jul 16 12:24:34.549639 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:34.549218 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" event={"ID":"b3abe92f-607b-4882-8e5f-5db96bf25b8f","Type":"ContainerDied","Data":"2063d4f1b06c8532af34fbe57edc75cd5fa2b1cda1e41d43bf06b5fcf4a4a1da"} Jul 16 12:24:35.674364 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:35.674344 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" Jul 16 12:24:35.785223 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:35.785188 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3abe92f-607b-4882-8e5f-5db96bf25b8f-util\") pod \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\" (UID: \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\") " Jul 16 12:24:35.785223 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:35.785225 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvjjm\" (UniqueName: \"kubernetes.io/projected/b3abe92f-607b-4882-8e5f-5db96bf25b8f-kube-api-access-fvjjm\") pod \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\" (UID: \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\") " Jul 16 12:24:35.785437 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:35.785250 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3abe92f-607b-4882-8e5f-5db96bf25b8f-bundle\") pod \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\" (UID: \"b3abe92f-607b-4882-8e5f-5db96bf25b8f\") " Jul 16 12:24:35.785993 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:35.785966 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3abe92f-607b-4882-8e5f-5db96bf25b8f-bundle" (OuterVolumeSpecName: "bundle") pod "b3abe92f-607b-4882-8e5f-5db96bf25b8f" (UID: "b3abe92f-607b-4882-8e5f-5db96bf25b8f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:24:35.787395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:35.787369 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3abe92f-607b-4882-8e5f-5db96bf25b8f-kube-api-access-fvjjm" (OuterVolumeSpecName: "kube-api-access-fvjjm") pod "b3abe92f-607b-4882-8e5f-5db96bf25b8f" (UID: "b3abe92f-607b-4882-8e5f-5db96bf25b8f"). InnerVolumeSpecName "kube-api-access-fvjjm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:24:35.790311 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:35.790273 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3abe92f-607b-4882-8e5f-5db96bf25b8f-util" (OuterVolumeSpecName: "util") pod "b3abe92f-607b-4882-8e5f-5db96bf25b8f" (UID: "b3abe92f-607b-4882-8e5f-5db96bf25b8f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:24:35.886569 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:35.886492 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvjjm\" (UniqueName: \"kubernetes.io/projected/b3abe92f-607b-4882-8e5f-5db96bf25b8f-kube-api-access-fvjjm\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:24:35.886569 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:35.886518 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3abe92f-607b-4882-8e5f-5db96bf25b8f-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:24:35.886569 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:35.886526 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3abe92f-607b-4882-8e5f-5db96bf25b8f-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:24:36.557101 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:36.557074 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" event={"ID":"b3abe92f-607b-4882-8e5f-5db96bf25b8f","Type":"ContainerDied","Data":"53edad9ebe1aefc61c09e738bc49b4ce881efb3d110a97f086fc1c13c09f1c6d"} Jul 16 12:24:36.557101 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:36.557097 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402wfvq" Jul 16 12:24:36.557283 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:36.557117 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53edad9ebe1aefc61c09e738bc49b4ce881efb3d110a97f086fc1c13c09f1c6d" Jul 16 12:24:41.570028 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.569994 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4"] Jul 16 12:24:41.570426 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.570229 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3abe92f-607b-4882-8e5f-5db96bf25b8f" containerName="extract" Jul 16 12:24:41.570426 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.570242 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3abe92f-607b-4882-8e5f-5db96bf25b8f" containerName="extract" Jul 16 12:24:41.570426 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.570258 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3abe92f-607b-4882-8e5f-5db96bf25b8f" containerName="pull" Jul 16 12:24:41.570426 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.570266 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3abe92f-607b-4882-8e5f-5db96bf25b8f" containerName="pull" Jul 16 12:24:41.570426 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.570279 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3abe92f-607b-4882-8e5f-5db96bf25b8f" containerName="util" Jul 16 12:24:41.570426 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.570287 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3abe92f-607b-4882-8e5f-5db96bf25b8f" containerName="util" Jul 16 12:24:41.570426 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.570379 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3abe92f-607b-4882-8e5f-5db96bf25b8f" containerName="extract" Jul 16 12:24:41.574689 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.574670 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4" Jul 16 12:24:41.577593 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.577577 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-cvsd9\"" Jul 16 12:24:41.577735 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.577616 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 16 12:24:41.577735 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.577621 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:24:41.589873 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.589854 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4"] Jul 16 12:24:41.730170 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.730138 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jkz4\" (UID: \"b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4" Jul 16 12:24:41.730360 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.730182 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlwfg\" (UniqueName: \"kubernetes.io/projected/b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5-kube-api-access-xlwfg\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jkz4\" (UID: \"b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4" Jul 16 12:24:41.831513 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.831422 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jkz4\" (UID: \"b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4" Jul 16 12:24:41.831513 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.831475 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlwfg\" (UniqueName: \"kubernetes.io/projected/b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5-kube-api-access-xlwfg\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jkz4\" (UID: \"b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4" Jul 16 12:24:41.831807 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.831781 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jkz4\" (UID: \"b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4" Jul 16 12:24:41.840523 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.840502 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlwfg\" (UniqueName: \"kubernetes.io/projected/b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5-kube-api-access-xlwfg\") pod \"cert-manager-operator-controller-manager-55c444bf86-8jkz4\" (UID: \"b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4" Jul 16 12:24:41.883249 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:41.883226 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4" Jul 16 12:24:42.017948 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:42.017924 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4"] Jul 16 12:24:42.020761 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:24:42.020733 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb31eeb0d_a0c5_4f59_81bc_c9746e6dbde5.slice/crio-562b4042e4a323b495dce387905f773272541f7885ec572a2b8245dd828933cd WatchSource:0}: Error finding container 562b4042e4a323b495dce387905f773272541f7885ec572a2b8245dd828933cd: Status 404 returned error can't find the container with id 562b4042e4a323b495dce387905f773272541f7885ec572a2b8245dd828933cd Jul 16 12:24:42.578605 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:42.578565 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4" event={"ID":"b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5","Type":"ContainerStarted","Data":"562b4042e4a323b495dce387905f773272541f7885ec572a2b8245dd828933cd"} Jul 16 12:24:43.584613 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:43.584574 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4" event={"ID":"b31eeb0d-a0c5-4f59-81bc-c9746e6dbde5","Type":"ContainerStarted","Data":"8eff7e5cfd7b55918def13ca975c55c80480e6ab475cc011d78e55c7a5a4f3dd"} Jul 16 12:24:43.619554 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:43.619486 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8jkz4" podStartSLOduration=1.218429496 podStartE2EDuration="2.619468865s" totalImagesPullingTime="1.401039369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:24:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:24:43.617451785 +0000 UTC m=+247.479353043" watchObservedRunningTime="2026-07-16 12:24:43.619468865 +0000 UTC m=+247.481370110" Jul 16 12:24:45.069529 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.069498 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k"] Jul 16 12:24:45.072824 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.072808 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" Jul 16 12:24:45.075571 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.075550 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 12:24:45.075688 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.075599 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 12:24:45.076599 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.076582 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-578t6\"" Jul 16 12:24:45.082813 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.082792 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k"] Jul 16 12:24:45.160089 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.160054 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfff3f48-8ab7-4de5-af19-14a313a384cd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k\" (UID: \"dfff3f48-8ab7-4de5-af19-14a313a384cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" Jul 16 12:24:45.160261 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.160093 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfff3f48-8ab7-4de5-af19-14a313a384cd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k\" (UID: \"dfff3f48-8ab7-4de5-af19-14a313a384cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" Jul 16 12:24:45.160261 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.160128 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4pm2\" (UniqueName: \"kubernetes.io/projected/dfff3f48-8ab7-4de5-af19-14a313a384cd-kube-api-access-k4pm2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k\" (UID: \"dfff3f48-8ab7-4de5-af19-14a313a384cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" Jul 16 12:24:45.261427 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.261390 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfff3f48-8ab7-4de5-af19-14a313a384cd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k\" (UID: \"dfff3f48-8ab7-4de5-af19-14a313a384cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" Jul 16 12:24:45.261427 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.261428 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfff3f48-8ab7-4de5-af19-14a313a384cd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k\" (UID: \"dfff3f48-8ab7-4de5-af19-14a313a384cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" Jul 16 12:24:45.261629 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.261450 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pm2\" (UniqueName: \"kubernetes.io/projected/dfff3f48-8ab7-4de5-af19-14a313a384cd-kube-api-access-k4pm2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k\" (UID: \"dfff3f48-8ab7-4de5-af19-14a313a384cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" Jul 16 12:24:45.261870 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.261849 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfff3f48-8ab7-4de5-af19-14a313a384cd-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k\" (UID: \"dfff3f48-8ab7-4de5-af19-14a313a384cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" Jul 16 12:24:45.261908 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.261856 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfff3f48-8ab7-4de5-af19-14a313a384cd-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k\" (UID: \"dfff3f48-8ab7-4de5-af19-14a313a384cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" Jul 16 12:24:45.278254 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.278224 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4pm2\" (UniqueName: \"kubernetes.io/projected/dfff3f48-8ab7-4de5-af19-14a313a384cd-kube-api-access-k4pm2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k\" (UID: \"dfff3f48-8ab7-4de5-af19-14a313a384cd\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" Jul 16 12:24:45.360978 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.360895 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nq8lk"] Jul 16 12:24:45.363977 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.363961 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nq8lk" Jul 16 12:24:45.366948 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.366927 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 12:24:45.367092 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.367008 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 12:24:45.367162 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.367150 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-9ttxn\"" Jul 16 12:24:45.376207 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.376181 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nq8lk"] Jul 16 12:24:45.381968 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.381947 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" Jul 16 12:24:45.463804 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.463774 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sh4v\" (UniqueName: \"kubernetes.io/projected/980012b9-6009-48c5-8f2f-7c0d1e7c18eb-kube-api-access-6sh4v\") pod \"cert-manager-webhook-6486ffd997-nq8lk\" (UID: \"980012b9-6009-48c5-8f2f-7c0d1e7c18eb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nq8lk" Jul 16 12:24:45.463955 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.463832 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/980012b9-6009-48c5-8f2f-7c0d1e7c18eb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nq8lk\" (UID: \"980012b9-6009-48c5-8f2f-7c0d1e7c18eb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nq8lk" Jul 16 12:24:45.511805 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.511776 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k"] Jul 16 12:24:45.514323 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:24:45.514279 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfff3f48_8ab7_4de5_af19_14a313a384cd.slice/crio-845e2fc507a09843de57df2c06693495b99f376c760f447f774daa50981367d4 WatchSource:0}: Error finding container 845e2fc507a09843de57df2c06693495b99f376c760f447f774daa50981367d4: Status 404 returned error can't find the container with id 845e2fc507a09843de57df2c06693495b99f376c760f447f774daa50981367d4 Jul 16 12:24:45.565024 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.565003 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sh4v\" (UniqueName: \"kubernetes.io/projected/980012b9-6009-48c5-8f2f-7c0d1e7c18eb-kube-api-access-6sh4v\") pod \"cert-manager-webhook-6486ffd997-nq8lk\" (UID: \"980012b9-6009-48c5-8f2f-7c0d1e7c18eb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nq8lk" Jul 16 12:24:45.565114 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.565043 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/980012b9-6009-48c5-8f2f-7c0d1e7c18eb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nq8lk\" (UID: \"980012b9-6009-48c5-8f2f-7c0d1e7c18eb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nq8lk" Jul 16 12:24:45.576955 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.576936 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sh4v\" (UniqueName: \"kubernetes.io/projected/980012b9-6009-48c5-8f2f-7c0d1e7c18eb-kube-api-access-6sh4v\") pod \"cert-manager-webhook-6486ffd997-nq8lk\" (UID: \"980012b9-6009-48c5-8f2f-7c0d1e7c18eb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nq8lk" Jul 16 12:24:45.577183 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.577167 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/980012b9-6009-48c5-8f2f-7c0d1e7c18eb-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-nq8lk\" (UID: \"980012b9-6009-48c5-8f2f-7c0d1e7c18eb\") " pod="cert-manager/cert-manager-webhook-6486ffd997-nq8lk" Jul 16 12:24:45.593482 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.593450 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" event={"ID":"dfff3f48-8ab7-4de5-af19-14a313a384cd","Type":"ContainerStarted","Data":"12601af47fbd094a9f0257c1906735d3f415fb59afbb632ba2d50a9635494f5a"} Jul 16 12:24:45.593605 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.593509 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" event={"ID":"dfff3f48-8ab7-4de5-af19-14a313a384cd","Type":"ContainerStarted","Data":"845e2fc507a09843de57df2c06693495b99f376c760f447f774daa50981367d4"} Jul 16 12:24:45.688574 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.688545 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-nq8lk" Jul 16 12:24:45.816756 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:45.816732 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-nq8lk"] Jul 16 12:24:45.818539 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:24:45.818507 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod980012b9_6009_48c5_8f2f_7c0d1e7c18eb.slice/crio-881fe8ddad2c1840112e70df266e98b72f1a4aaf2c41934b40eedf343befa0b6 WatchSource:0}: Error finding container 881fe8ddad2c1840112e70df266e98b72f1a4aaf2c41934b40eedf343befa0b6: Status 404 returned error can't find the container with id 881fe8ddad2c1840112e70df266e98b72f1a4aaf2c41934b40eedf343befa0b6 Jul 16 12:24:46.599780 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:46.599703 2577 generic.go:356] "Generic (PLEG): container finished" podID="dfff3f48-8ab7-4de5-af19-14a313a384cd" containerID="12601af47fbd094a9f0257c1906735d3f415fb59afbb632ba2d50a9635494f5a" exitCode=0 Jul 16 12:24:46.600196 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:46.599802 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" event={"ID":"dfff3f48-8ab7-4de5-af19-14a313a384cd","Type":"ContainerDied","Data":"12601af47fbd094a9f0257c1906735d3f415fb59afbb632ba2d50a9635494f5a"} Jul 16 12:24:46.601332 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:46.601092 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nq8lk" event={"ID":"980012b9-6009-48c5-8f2f-7c0d1e7c18eb","Type":"ContainerStarted","Data":"881fe8ddad2c1840112e70df266e98b72f1a4aaf2c41934b40eedf343befa0b6"} Jul 16 12:24:49.615981 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:49.615944 2577 generic.go:356] "Generic (PLEG): container finished" podID="dfff3f48-8ab7-4de5-af19-14a313a384cd" containerID="59b6d3549fc83edde30302bd73c3c71b6d53c80953800adc3ce21ba64409c9c8" exitCode=0 Jul 16 12:24:49.616435 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:49.616026 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" event={"ID":"dfff3f48-8ab7-4de5-af19-14a313a384cd","Type":"ContainerDied","Data":"59b6d3549fc83edde30302bd73c3c71b6d53c80953800adc3ce21ba64409c9c8"} Jul 16 12:24:49.617725 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:49.617704 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-nq8lk" event={"ID":"980012b9-6009-48c5-8f2f-7c0d1e7c18eb","Type":"ContainerStarted","Data":"14c2e0cb2410883bd929210728577bc097f2b0a7a65becbe9fd9b23b2ec7e856"} Jul 16 12:24:49.617842 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:49.617831 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nq8lk" Jul 16 12:24:49.648696 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:49.648654 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-nq8lk" podStartSLOduration=1.506279084 podStartE2EDuration="4.648638791s" totalImagesPullingTime="3.142359707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:24:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:24:49.646923557 +0000 UTC m=+253.508824805" watchObservedRunningTime="2026-07-16 12:24:49.648638791 +0000 UTC m=+253.510540038" Jul 16 12:24:50.623373 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:50.623344 2577 generic.go:356] "Generic (PLEG): container finished" podID="dfff3f48-8ab7-4de5-af19-14a313a384cd" containerID="1ac0232669b76b4e02444362a12e7892dac28de4ac60a6c48b2937c2ba3132cd" exitCode=0 Jul 16 12:24:50.623838 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:50.623423 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" event={"ID":"dfff3f48-8ab7-4de5-af19-14a313a384cd","Type":"ContainerDied","Data":"1ac0232669b76b4e02444362a12e7892dac28de4ac60a6c48b2937c2ba3132cd"} Jul 16 12:24:51.754759 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:51.754733 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" Jul 16 12:24:51.915846 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:51.915822 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4pm2\" (UniqueName: \"kubernetes.io/projected/dfff3f48-8ab7-4de5-af19-14a313a384cd-kube-api-access-k4pm2\") pod \"dfff3f48-8ab7-4de5-af19-14a313a384cd\" (UID: \"dfff3f48-8ab7-4de5-af19-14a313a384cd\") " Jul 16 12:24:51.916031 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:51.915866 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfff3f48-8ab7-4de5-af19-14a313a384cd-bundle\") pod \"dfff3f48-8ab7-4de5-af19-14a313a384cd\" (UID: \"dfff3f48-8ab7-4de5-af19-14a313a384cd\") " Jul 16 12:24:51.916031 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:51.915900 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfff3f48-8ab7-4de5-af19-14a313a384cd-util\") pod \"dfff3f48-8ab7-4de5-af19-14a313a384cd\" (UID: \"dfff3f48-8ab7-4de5-af19-14a313a384cd\") " Jul 16 12:24:51.916266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:51.916239 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfff3f48-8ab7-4de5-af19-14a313a384cd-bundle" (OuterVolumeSpecName: "bundle") pod "dfff3f48-8ab7-4de5-af19-14a313a384cd" (UID: "dfff3f48-8ab7-4de5-af19-14a313a384cd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:24:51.918048 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:51.918020 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfff3f48-8ab7-4de5-af19-14a313a384cd-kube-api-access-k4pm2" (OuterVolumeSpecName: "kube-api-access-k4pm2") pod "dfff3f48-8ab7-4de5-af19-14a313a384cd" (UID: "dfff3f48-8ab7-4de5-af19-14a313a384cd"). InnerVolumeSpecName "kube-api-access-k4pm2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:24:51.920428 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:51.920409 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfff3f48-8ab7-4de5-af19-14a313a384cd-util" (OuterVolumeSpecName: "util") pod "dfff3f48-8ab7-4de5-af19-14a313a384cd" (UID: "dfff3f48-8ab7-4de5-af19-14a313a384cd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:24:52.016945 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:52.016916 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4pm2\" (UniqueName: \"kubernetes.io/projected/dfff3f48-8ab7-4de5-af19-14a313a384cd-kube-api-access-k4pm2\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:24:52.016945 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:52.016942 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfff3f48-8ab7-4de5-af19-14a313a384cd-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:24:52.017085 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:52.016955 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfff3f48-8ab7-4de5-af19-14a313a384cd-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:24:52.632995 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:52.632970 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" Jul 16 12:24:52.637359 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:52.637327 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7f47k" event={"ID":"dfff3f48-8ab7-4de5-af19-14a313a384cd","Type":"ContainerDied","Data":"845e2fc507a09843de57df2c06693495b99f376c760f447f774daa50981367d4"} Jul 16 12:24:52.637514 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:52.637376 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="845e2fc507a09843de57df2c06693495b99f376c760f447f774daa50981367d4" Jul 16 12:24:55.626033 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:55.626001 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-nq8lk" Jul 16 12:24:56.515277 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.515245 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fsrwn"] Jul 16 12:24:56.515514 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.515501 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfff3f48-8ab7-4de5-af19-14a313a384cd" containerName="pull" Jul 16 12:24:56.515514 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.515514 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfff3f48-8ab7-4de5-af19-14a313a384cd" containerName="pull" Jul 16 12:24:56.515600 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.515524 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfff3f48-8ab7-4de5-af19-14a313a384cd" containerName="extract" Jul 16 12:24:56.515600 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.515530 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfff3f48-8ab7-4de5-af19-14a313a384cd" containerName="extract" Jul 16 12:24:56.515600 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.515539 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfff3f48-8ab7-4de5-af19-14a313a384cd" containerName="util" Jul 16 12:24:56.515600 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.515544 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfff3f48-8ab7-4de5-af19-14a313a384cd" containerName="util" Jul 16 12:24:56.515600 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.515588 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfff3f48-8ab7-4de5-af19-14a313a384cd" containerName="extract" Jul 16 12:24:56.519305 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.519278 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fsrwn" Jul 16 12:24:56.521587 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.521571 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-qkcdb\"" Jul 16 12:24:56.528597 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.528578 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fsrwn"] Jul 16 12:24:56.551344 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.551315 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt2rj\" (UniqueName: \"kubernetes.io/projected/9e06a6f0-d568-408e-a92a-6338be38b2b4-kube-api-access-jt2rj\") pod \"cert-manager-55b4d954d5-fsrwn\" (UID: \"9e06a6f0-d568-408e-a92a-6338be38b2b4\") " pod="cert-manager/cert-manager-55b4d954d5-fsrwn" Jul 16 12:24:56.551453 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.551369 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e06a6f0-d568-408e-a92a-6338be38b2b4-bound-sa-token\") pod \"cert-manager-55b4d954d5-fsrwn\" (UID: \"9e06a6f0-d568-408e-a92a-6338be38b2b4\") " pod="cert-manager/cert-manager-55b4d954d5-fsrwn" Jul 16 12:24:56.651992 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.651959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt2rj\" (UniqueName: \"kubernetes.io/projected/9e06a6f0-d568-408e-a92a-6338be38b2b4-kube-api-access-jt2rj\") pod \"cert-manager-55b4d954d5-fsrwn\" (UID: \"9e06a6f0-d568-408e-a92a-6338be38b2b4\") " pod="cert-manager/cert-manager-55b4d954d5-fsrwn" Jul 16 12:24:56.652433 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.652013 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e06a6f0-d568-408e-a92a-6338be38b2b4-bound-sa-token\") pod \"cert-manager-55b4d954d5-fsrwn\" (UID: \"9e06a6f0-d568-408e-a92a-6338be38b2b4\") " pod="cert-manager/cert-manager-55b4d954d5-fsrwn" Jul 16 12:24:56.660141 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.660119 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e06a6f0-d568-408e-a92a-6338be38b2b4-bound-sa-token\") pod \"cert-manager-55b4d954d5-fsrwn\" (UID: \"9e06a6f0-d568-408e-a92a-6338be38b2b4\") " pod="cert-manager/cert-manager-55b4d954d5-fsrwn" Jul 16 12:24:56.660375 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.660358 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt2rj\" (UniqueName: \"kubernetes.io/projected/9e06a6f0-d568-408e-a92a-6338be38b2b4-kube-api-access-jt2rj\") pod \"cert-manager-55b4d954d5-fsrwn\" (UID: \"9e06a6f0-d568-408e-a92a-6338be38b2b4\") " pod="cert-manager/cert-manager-55b4d954d5-fsrwn" Jul 16 12:24:56.828721 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.828649 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-fsrwn" Jul 16 12:24:56.951440 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:56.951411 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-fsrwn"] Jul 16 12:24:56.954330 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:24:56.954285 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e06a6f0_d568_408e_a92a_6338be38b2b4.slice/crio-623d541ca21f8907fc5a835d36a98cba3869a9a1ca5e82d72d25f804fe185779 WatchSource:0}: Error finding container 623d541ca21f8907fc5a835d36a98cba3869a9a1ca5e82d72d25f804fe185779: Status 404 returned error can't find the container with id 623d541ca21f8907fc5a835d36a98cba3869a9a1ca5e82d72d25f804fe185779 Jul 16 12:24:57.657627 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:57.657549 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fsrwn" event={"ID":"9e06a6f0-d568-408e-a92a-6338be38b2b4","Type":"ContainerStarted","Data":"d5d09da6e0ca2dc7e05942a305109b725ef2355d6fdde4ae94ad3e470666acf8"} Jul 16 12:24:57.657627 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:57.657593 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-fsrwn" event={"ID":"9e06a6f0-d568-408e-a92a-6338be38b2b4","Type":"ContainerStarted","Data":"623d541ca21f8907fc5a835d36a98cba3869a9a1ca5e82d72d25f804fe185779"} Jul 16 12:24:57.673844 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:24:57.673795 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-fsrwn" podStartSLOduration=1.673780974 podStartE2EDuration="1.673780974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:24:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:24:57.673131013 +0000 UTC m=+261.535032262" watchObservedRunningTime="2026-07-16 12:24:57.673780974 +0000 UTC m=+261.535682219" Jul 16 12:25:03.010831 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.010795 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj"] Jul 16 12:25:03.084636 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.084604 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj"] Jul 16 12:25:03.084795 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.084708 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj" Jul 16 12:25:03.087043 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.087019 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-xgld6\"" Jul 16 12:25:03.087197 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.087115 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 12:25:03.087265 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.087231 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 12:25:03.089696 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.089673 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfs4j\" (UniqueName: \"kubernetes.io/projected/d85e0849-773f-4f4a-a2b1-4cf787b34452-kube-api-access-sfs4j\") pod \"openshift-lws-operator-fd8ccff4c-rx9jj\" (UID: \"d85e0849-773f-4f4a-a2b1-4cf787b34452\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj" Jul 16 12:25:03.089781 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.089727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d85e0849-773f-4f4a-a2b1-4cf787b34452-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rx9jj\" (UID: \"d85e0849-773f-4f4a-a2b1-4cf787b34452\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj" Jul 16 12:25:03.190141 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.190102 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfs4j\" (UniqueName: \"kubernetes.io/projected/d85e0849-773f-4f4a-a2b1-4cf787b34452-kube-api-access-sfs4j\") pod \"openshift-lws-operator-fd8ccff4c-rx9jj\" (UID: \"d85e0849-773f-4f4a-a2b1-4cf787b34452\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj" Jul 16 12:25:03.190363 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.190154 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d85e0849-773f-4f4a-a2b1-4cf787b34452-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rx9jj\" (UID: \"d85e0849-773f-4f4a-a2b1-4cf787b34452\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj" Jul 16 12:25:03.190623 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.190605 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d85e0849-773f-4f4a-a2b1-4cf787b34452-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rx9jj\" (UID: \"d85e0849-773f-4f4a-a2b1-4cf787b34452\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj" Jul 16 12:25:03.199421 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.199393 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfs4j\" (UniqueName: \"kubernetes.io/projected/d85e0849-773f-4f4a-a2b1-4cf787b34452-kube-api-access-sfs4j\") pod \"openshift-lws-operator-fd8ccff4c-rx9jj\" (UID: \"d85e0849-773f-4f4a-a2b1-4cf787b34452\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj" Jul 16 12:25:03.394604 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.394514 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj" Jul 16 12:25:03.541513 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.541491 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj"] Jul 16 12:25:03.543637 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:25:03.543607 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd85e0849_773f_4f4a_a2b1_4cf787b34452.slice/crio-bca095177090e846b1749e6c2402799f29e635f4f6535c1580507c289df16c06 WatchSource:0}: Error finding container bca095177090e846b1749e6c2402799f29e635f4f6535c1580507c289df16c06: Status 404 returned error can't find the container with id bca095177090e846b1749e6c2402799f29e635f4f6535c1580507c289df16c06 Jul 16 12:25:03.680403 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:03.680374 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj" event={"ID":"d85e0849-773f-4f4a-a2b1-4cf787b34452","Type":"ContainerStarted","Data":"bca095177090e846b1749e6c2402799f29e635f4f6535c1580507c289df16c06"} Jul 16 12:25:06.694005 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:06.693973 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj" event={"ID":"d85e0849-773f-4f4a-a2b1-4cf787b34452","Type":"ContainerStarted","Data":"cfd05ccbf8c0b5277e2090208495267007e2ea4758d72037c9387a0be307532f"} Jul 16 12:25:06.717174 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:06.717120 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rx9jj" podStartSLOduration=2.644229239 podStartE2EDuration="4.717104828s" totalImagesPullingTime="2.072875589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:25:06.715462082 +0000 UTC m=+270.577363328" watchObservedRunningTime="2026-07-16 12:25:06.717104828 +0000 UTC m=+270.579006075" Jul 16 12:25:11.887529 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:11.887491 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv"] Jul 16 12:25:11.890865 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:11.890848 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" Jul 16 12:25:11.893087 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:11.893064 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 12:25:11.893900 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:11.893884 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 12:25:11.893971 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:11.893905 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-578t6\"" Jul 16 12:25:11.899769 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:11.899748 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv"] Jul 16 12:25:11.959131 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:11.959095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a06b07da-e7c6-4eff-8432-1c94e2b65b38-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv\" (UID: \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" Jul 16 12:25:11.959327 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:11.959152 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9zll\" (UniqueName: \"kubernetes.io/projected/a06b07da-e7c6-4eff-8432-1c94e2b65b38-kube-api-access-f9zll\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv\" (UID: \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" Jul 16 12:25:11.959327 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:11.959173 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a06b07da-e7c6-4eff-8432-1c94e2b65b38-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv\" (UID: \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" Jul 16 12:25:12.060144 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:12.060103 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a06b07da-e7c6-4eff-8432-1c94e2b65b38-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv\" (UID: \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" Jul 16 12:25:12.060398 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:12.060159 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9zll\" (UniqueName: \"kubernetes.io/projected/a06b07da-e7c6-4eff-8432-1c94e2b65b38-kube-api-access-f9zll\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv\" (UID: \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" Jul 16 12:25:12.060398 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:12.060179 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a06b07da-e7c6-4eff-8432-1c94e2b65b38-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv\" (UID: \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" Jul 16 12:25:12.060617 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:12.060596 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a06b07da-e7c6-4eff-8432-1c94e2b65b38-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv\" (UID: \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" Jul 16 12:25:12.060680 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:12.060658 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a06b07da-e7c6-4eff-8432-1c94e2b65b38-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv\" (UID: \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" Jul 16 12:25:12.068841 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:12.068812 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9zll\" (UniqueName: \"kubernetes.io/projected/a06b07da-e7c6-4eff-8432-1c94e2b65b38-kube-api-access-f9zll\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv\" (UID: \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" Jul 16 12:25:12.199729 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:12.199693 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" Jul 16 12:25:12.354700 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:12.354661 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv"] Jul 16 12:25:12.359315 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:25:12.359267 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda06b07da_e7c6_4eff_8432_1c94e2b65b38.slice/crio-edbd51de6e1cceb072a7bb3a369a24b22f74db17298e985d451a1cd6189c9f1e WatchSource:0}: Error finding container edbd51de6e1cceb072a7bb3a369a24b22f74db17298e985d451a1cd6189c9f1e: Status 404 returned error can't find the container with id edbd51de6e1cceb072a7bb3a369a24b22f74db17298e985d451a1cd6189c9f1e Jul 16 12:25:12.718948 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:12.718856 2577 generic.go:356] "Generic (PLEG): container finished" podID="a06b07da-e7c6-4eff-8432-1c94e2b65b38" containerID="046fe4dc2c23f4d4afc8eaaeb2712e3e59a2841cace4866c48de13e57b095b2f" exitCode=0 Jul 16 12:25:12.718948 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:12.718934 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" event={"ID":"a06b07da-e7c6-4eff-8432-1c94e2b65b38","Type":"ContainerDied","Data":"046fe4dc2c23f4d4afc8eaaeb2712e3e59a2841cace4866c48de13e57b095b2f"} Jul 16 12:25:12.719178 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:12.718980 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" event={"ID":"a06b07da-e7c6-4eff-8432-1c94e2b65b38","Type":"ContainerStarted","Data":"edbd51de6e1cceb072a7bb3a369a24b22f74db17298e985d451a1cd6189c9f1e"} Jul 16 12:25:13.724876 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:13.724839 2577 generic.go:356] "Generic (PLEG): container finished" podID="a06b07da-e7c6-4eff-8432-1c94e2b65b38" containerID="0b20102a1d087a526ecf796145c59942ac3be10ac66a92ad79d1bd139a26dc40" exitCode=0 Jul 16 12:25:13.725266 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:13.724893 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" event={"ID":"a06b07da-e7c6-4eff-8432-1c94e2b65b38","Type":"ContainerDied","Data":"0b20102a1d087a526ecf796145c59942ac3be10ac66a92ad79d1bd139a26dc40"} Jul 16 12:25:14.730498 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:14.730468 2577 generic.go:356] "Generic (PLEG): container finished" podID="a06b07da-e7c6-4eff-8432-1c94e2b65b38" containerID="1cbc0a45c31cb95580c96d4f979f97d2293b774d5f21b48740f6ae94ab7ea25d" exitCode=0 Jul 16 12:25:14.730876 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:14.730526 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" event={"ID":"a06b07da-e7c6-4eff-8432-1c94e2b65b38","Type":"ContainerDied","Data":"1cbc0a45c31cb95580c96d4f979f97d2293b774d5f21b48740f6ae94ab7ea25d"} Jul 16 12:25:15.856365 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:15.856336 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" Jul 16 12:25:15.888818 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:15.888789 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a06b07da-e7c6-4eff-8432-1c94e2b65b38-bundle\") pod \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\" (UID: \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\") " Jul 16 12:25:15.888818 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:15.888825 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9zll\" (UniqueName: \"kubernetes.io/projected/a06b07da-e7c6-4eff-8432-1c94e2b65b38-kube-api-access-f9zll\") pod \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\" (UID: \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\") " Jul 16 12:25:15.889053 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:15.888882 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a06b07da-e7c6-4eff-8432-1c94e2b65b38-util\") pod \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\" (UID: \"a06b07da-e7c6-4eff-8432-1c94e2b65b38\") " Jul 16 12:25:15.889589 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:15.889552 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06b07da-e7c6-4eff-8432-1c94e2b65b38-bundle" (OuterVolumeSpecName: "bundle") pod "a06b07da-e7c6-4eff-8432-1c94e2b65b38" (UID: "a06b07da-e7c6-4eff-8432-1c94e2b65b38"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:25:15.891093 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:15.891065 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a06b07da-e7c6-4eff-8432-1c94e2b65b38-kube-api-access-f9zll" (OuterVolumeSpecName: "kube-api-access-f9zll") pod "a06b07da-e7c6-4eff-8432-1c94e2b65b38" (UID: "a06b07da-e7c6-4eff-8432-1c94e2b65b38"). InnerVolumeSpecName "kube-api-access-f9zll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:25:15.897740 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:15.897713 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a06b07da-e7c6-4eff-8432-1c94e2b65b38-util" (OuterVolumeSpecName: "util") pod "a06b07da-e7c6-4eff-8432-1c94e2b65b38" (UID: "a06b07da-e7c6-4eff-8432-1c94e2b65b38"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:25:15.990316 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:15.990195 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a06b07da-e7c6-4eff-8432-1c94e2b65b38-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:25:15.990316 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:15.990231 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9zll\" (UniqueName: \"kubernetes.io/projected/a06b07da-e7c6-4eff-8432-1c94e2b65b38-kube-api-access-f9zll\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:25:15.990316 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:15.990241 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a06b07da-e7c6-4eff-8432-1c94e2b65b38-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:25:16.740804 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:16.740775 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" event={"ID":"a06b07da-e7c6-4eff-8432-1c94e2b65b38","Type":"ContainerDied","Data":"edbd51de6e1cceb072a7bb3a369a24b22f74db17298e985d451a1cd6189c9f1e"} Jul 16 12:25:16.740804 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:16.740814 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afg6kxv" Jul 16 12:25:16.741007 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:16.740819 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="edbd51de6e1cceb072a7bb3a369a24b22f74db17298e985d451a1cd6189c9f1e" Jul 16 12:25:22.046763 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.046731 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c"] Jul 16 12:25:22.047144 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.046972 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06b07da-e7c6-4eff-8432-1c94e2b65b38" containerName="util" Jul 16 12:25:22.047144 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.046982 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06b07da-e7c6-4eff-8432-1c94e2b65b38" containerName="util" Jul 16 12:25:22.047144 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.046994 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06b07da-e7c6-4eff-8432-1c94e2b65b38" containerName="pull" Jul 16 12:25:22.047144 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.046999 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06b07da-e7c6-4eff-8432-1c94e2b65b38" containerName="pull" Jul 16 12:25:22.047144 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.047012 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a06b07da-e7c6-4eff-8432-1c94e2b65b38" containerName="extract" Jul 16 12:25:22.047144 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.047018 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a06b07da-e7c6-4eff-8432-1c94e2b65b38" containerName="extract" Jul 16 12:25:22.047144 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.047060 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a06b07da-e7c6-4eff-8432-1c94e2b65b38" containerName="extract" Jul 16 12:25:22.050885 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.050866 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.055201 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.055177 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 12:25:22.055342 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.055202 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-grdmx\"" Jul 16 12:25:22.055342 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.055177 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 12:25:22.055342 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.055177 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 12:25:22.060556 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.060539 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c"] Jul 16 12:25:22.136999 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.136969 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw8zz\" (UniqueName: \"kubernetes.io/projected/827f5bf7-5ca8-4c59-9303-5556bb580e83-kube-api-access-cw8zz\") pod \"lws-controller-manager-698fdd4469-vzv4c\" (UID: \"827f5bf7-5ca8-4c59-9303-5556bb580e83\") " pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.137129 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.137019 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/827f5bf7-5ca8-4c59-9303-5556bb580e83-manager-config\") pod \"lws-controller-manager-698fdd4469-vzv4c\" (UID: \"827f5bf7-5ca8-4c59-9303-5556bb580e83\") " pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.137129 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.137073 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/827f5bf7-5ca8-4c59-9303-5556bb580e83-metrics-cert\") pod \"lws-controller-manager-698fdd4469-vzv4c\" (UID: \"827f5bf7-5ca8-4c59-9303-5556bb580e83\") " pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.137129 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.137097 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/827f5bf7-5ca8-4c59-9303-5556bb580e83-cert\") pod \"lws-controller-manager-698fdd4469-vzv4c\" (UID: \"827f5bf7-5ca8-4c59-9303-5556bb580e83\") " pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.238280 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.238245 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/827f5bf7-5ca8-4c59-9303-5556bb580e83-cert\") pod \"lws-controller-manager-698fdd4469-vzv4c\" (UID: \"827f5bf7-5ca8-4c59-9303-5556bb580e83\") " pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.238450 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.238318 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw8zz\" (UniqueName: \"kubernetes.io/projected/827f5bf7-5ca8-4c59-9303-5556bb580e83-kube-api-access-cw8zz\") pod \"lws-controller-manager-698fdd4469-vzv4c\" (UID: \"827f5bf7-5ca8-4c59-9303-5556bb580e83\") " pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.238450 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.238354 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/827f5bf7-5ca8-4c59-9303-5556bb580e83-manager-config\") pod \"lws-controller-manager-698fdd4469-vzv4c\" (UID: \"827f5bf7-5ca8-4c59-9303-5556bb580e83\") " pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.238450 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.238386 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/827f5bf7-5ca8-4c59-9303-5556bb580e83-metrics-cert\") pod \"lws-controller-manager-698fdd4469-vzv4c\" (UID: \"827f5bf7-5ca8-4c59-9303-5556bb580e83\") " pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.239005 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.238978 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/827f5bf7-5ca8-4c59-9303-5556bb580e83-manager-config\") pod \"lws-controller-manager-698fdd4469-vzv4c\" (UID: \"827f5bf7-5ca8-4c59-9303-5556bb580e83\") " pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.240948 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.240924 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/827f5bf7-5ca8-4c59-9303-5556bb580e83-metrics-cert\") pod \"lws-controller-manager-698fdd4469-vzv4c\" (UID: \"827f5bf7-5ca8-4c59-9303-5556bb580e83\") " pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.241011 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.240994 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/827f5bf7-5ca8-4c59-9303-5556bb580e83-cert\") pod \"lws-controller-manager-698fdd4469-vzv4c\" (UID: \"827f5bf7-5ca8-4c59-9303-5556bb580e83\") " pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.249363 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.249345 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw8zz\" (UniqueName: \"kubernetes.io/projected/827f5bf7-5ca8-4c59-9303-5556bb580e83-kube-api-access-cw8zz\") pod \"lws-controller-manager-698fdd4469-vzv4c\" (UID: \"827f5bf7-5ca8-4c59-9303-5556bb580e83\") " pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.359508 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.359426 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:22.481606 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.481574 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c"] Jul 16 12:25:22.484284 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:25:22.484249 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod827f5bf7_5ca8_4c59_9303_5556bb580e83.slice/crio-65aa3cf4681d97c1e5f779ac761bb04662fbc6d7051723445c3b4ebfe6c17edc WatchSource:0}: Error finding container 65aa3cf4681d97c1e5f779ac761bb04662fbc6d7051723445c3b4ebfe6c17edc: Status 404 returned error can't find the container with id 65aa3cf4681d97c1e5f779ac761bb04662fbc6d7051723445c3b4ebfe6c17edc Jul 16 12:25:22.768399 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:22.768362 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" event={"ID":"827f5bf7-5ca8-4c59-9303-5556bb580e83","Type":"ContainerStarted","Data":"65aa3cf4681d97c1e5f779ac761bb04662fbc6d7051723445c3b4ebfe6c17edc"} Jul 16 12:25:23.544846 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.544810 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l"] Jul 16 12:25:23.548611 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.548586 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.551521 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.551493 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 12:25:23.551645 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.551559 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rxvlt\"" Jul 16 12:25:23.554168 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.554147 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 12:25:23.555704 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.555615 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 12:25:23.557784 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.557759 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 12:25:23.580334 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.580305 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l"] Jul 16 12:25:23.649920 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.649883 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2224488-af18-4b2f-88cf-201c9a2c1c1b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.650092 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.649923 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2224488-af18-4b2f-88cf-201c9a2c1c1b-tmp\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.650092 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.649961 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2224488-af18-4b2f-88cf-201c9a2c1c1b-manifests\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.650092 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.649982 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2224488-af18-4b2f-88cf-201c9a2c1c1b-webhook-cert\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.650092 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.650055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g95x\" (UniqueName: \"kubernetes.io/projected/f2224488-af18-4b2f-88cf-201c9a2c1c1b-kube-api-access-2g95x\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.750423 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.750398 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2224488-af18-4b2f-88cf-201c9a2c1c1b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.750542 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.750431 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2224488-af18-4b2f-88cf-201c9a2c1c1b-tmp\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.750542 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.750460 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2224488-af18-4b2f-88cf-201c9a2c1c1b-manifests\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.750542 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.750476 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2224488-af18-4b2f-88cf-201c9a2c1c1b-webhook-cert\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.750542 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.750535 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g95x\" (UniqueName: \"kubernetes.io/projected/f2224488-af18-4b2f-88cf-201c9a2c1c1b-kube-api-access-2g95x\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.750805 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.750783 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2224488-af18-4b2f-88cf-201c9a2c1c1b-manifests\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.752950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.752906 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2224488-af18-4b2f-88cf-201c9a2c1c1b-tmp\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.753130 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.753108 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2224488-af18-4b2f-88cf-201c9a2c1c1b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.753223 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.753208 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2224488-af18-4b2f-88cf-201c9a2c1c1b-webhook-cert\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.760111 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.760082 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g95x\" (UniqueName: \"kubernetes.io/projected/f2224488-af18-4b2f-88cf-201c9a2c1c1b-kube-api-access-2g95x\") pod \"opendatahub-operator-controller-manager-87d97db45-g8k6l\" (UID: \"f2224488-af18-4b2f-88cf-201c9a2c1c1b\") " pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:23.778382 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.778345 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" event={"ID":"827f5bf7-5ca8-4c59-9303-5556bb580e83","Type":"ContainerStarted","Data":"61117388e5e2037aad31cafc64e35a457e6fec5bb0c1af466d9e667e47c54001"} Jul 16 12:25:23.778550 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.778476 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:23.799518 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.799431 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" podStartSLOduration=0.595502382 podStartE2EDuration="1.799414179s" totalImagesPullingTime="1.203911797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:25:23.798256737 +0000 UTC m=+287.660158021" watchObservedRunningTime="2026-07-16 12:25:23.799414179 +0000 UTC m=+287.661315423" Jul 16 12:25:23.861329 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:23.861278 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:24.001625 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:24.001604 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l"] Jul 16 12:25:24.003877 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:25:24.003850 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2224488_af18_4b2f_88cf_201c9a2c1c1b.slice/crio-c39fc6d82e9d5099f78a9e667889a996daf229e8c7f2c9c4b7b5b9d5464add5c WatchSource:0}: Error finding container c39fc6d82e9d5099f78a9e667889a996daf229e8c7f2c9c4b7b5b9d5464add5c: Status 404 returned error can't find the container with id c39fc6d82e9d5099f78a9e667889a996daf229e8c7f2c9c4b7b5b9d5464add5c Jul 16 12:25:24.784698 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:24.784663 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" event={"ID":"f2224488-af18-4b2f-88cf-201c9a2c1c1b","Type":"ContainerStarted","Data":"c39fc6d82e9d5099f78a9e667889a996daf229e8c7f2c9c4b7b5b9d5464add5c"} Jul 16 12:25:27.800940 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:27.800902 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" event={"ID":"f2224488-af18-4b2f-88cf-201c9a2c1c1b","Type":"ContainerStarted","Data":"77fea3bec396e1d3853b6c811c4181e628ad01b641a01fae385d8e33fd4c17f7"} Jul 16 12:25:28.806172 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:28.806135 2577 generic.go:356] "Generic (PLEG): container finished" podID="f2224488-af18-4b2f-88cf-201c9a2c1c1b" containerID="77fea3bec396e1d3853b6c811c4181e628ad01b641a01fae385d8e33fd4c17f7" exitCode=0 Jul 16 12:25:28.806620 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:28.806218 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" event={"ID":"f2224488-af18-4b2f-88cf-201c9a2c1c1b","Type":"ContainerDied","Data":"77fea3bec396e1d3853b6c811c4181e628ad01b641a01fae385d8e33fd4c17f7"} Jul 16 12:25:29.812852 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:29.812815 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" event={"ID":"f2224488-af18-4b2f-88cf-201c9a2c1c1b","Type":"ContainerStarted","Data":"b0541682057aaeab2006e27015d7a9d4460dba1e79a1be5706946d198dd2ca6a"} Jul 16 12:25:29.813205 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:29.812926 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:29.836434 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:29.836379 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" podStartSLOduration=2.27850345 podStartE2EDuration="6.836360815s" totalImagesPullingTime="3.505864238s" totalInitContainerRuntime="1.051993127s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:25:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:25:29.835855998 +0000 UTC m=+293.697757244" watchObservedRunningTime="2026-07-16 12:25:29.836360815 +0000 UTC m=+293.698262060" Jul 16 12:25:34.786879 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:34.786841 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-698fdd4469-vzv4c" Jul 16 12:25:36.484346 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:36.484311 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:25:36.485518 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:36.485487 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:25:36.496218 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:36.496198 2577 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 12:25:40.818790 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:40.818759 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-87d97db45-g8k6l" Jul 16 12:25:50.798968 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.798939 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764"] Jul 16 12:25:50.802764 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.802748 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.805598 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.805574 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 12:25:50.805598 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.805595 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 12:25:50.805794 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.805630 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 12:25:50.805794 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.805631 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qrnxr\"" Jul 16 12:25:50.813793 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.813773 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764"] Jul 16 12:25:50.877848 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.877819 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8vp2\" (UniqueName: \"kubernetes.io/projected/44f90567-cca7-491d-8c6f-a08ce954afb7-kube-api-access-z8vp2\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.878033 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.877860 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.878033 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.877923 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.878033 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.877966 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44f90567-cca7-491d-8c6f-a08ce954afb7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.878033 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.877998 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44f90567-cca7-491d-8c6f-a08ce954afb7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.878033 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.878026 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.878279 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.878053 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44f90567-cca7-491d-8c6f-a08ce954afb7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.878279 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.878139 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.878279 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.878183 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.959067 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.959036 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt"] Jul 16 12:25:50.962717 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.962701 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:50.965126 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.965105 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 12:25:50.965126 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.965123 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 12:25:50.965326 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.965219 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 12:25:50.965326 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.965234 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hqjbr\"" Jul 16 12:25:50.975747 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.975727 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt"] Jul 16 12:25:50.979383 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.979362 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.979463 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.979401 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.979463 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.979428 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44f90567-cca7-491d-8c6f-a08ce954afb7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.979626 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.979600 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44f90567-cca7-491d-8c6f-a08ce954afb7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.979694 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.979647 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.979694 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.979675 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44f90567-cca7-491d-8c6f-a08ce954afb7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.979792 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.979723 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.979792 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.979768 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.979792 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.979769 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.979939 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.979821 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.979939 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.979831 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8vp2\" (UniqueName: \"kubernetes.io/projected/44f90567-cca7-491d-8c6f-a08ce954afb7-kube-api-access-z8vp2\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.980039 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.979952 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.980145 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.980120 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.980651 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.980632 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44f90567-cca7-491d-8c6f-a08ce954afb7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.981934 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.981914 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44f90567-cca7-491d-8c6f-a08ce954afb7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.982010 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.981931 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44f90567-cca7-491d-8c6f-a08ce954afb7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.989585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.989559 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44f90567-cca7-491d-8c6f-a08ce954afb7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:50.990028 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:50.990011 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8vp2\" (UniqueName: \"kubernetes.io/projected/44f90567-cca7-491d-8c6f-a08ce954afb7-kube-api-access-z8vp2\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678nl764\" (UID: \"44f90567-cca7-491d-8c6f-a08ce954afb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:51.080775 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.080682 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9483cf16-7c69-4ace-9ba7-087053569a6a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.080775 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.080727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9483cf16-7c69-4ace-9ba7-087053569a6a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.080993 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.080794 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9483cf16-7c69-4ace-9ba7-087053569a6a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.080993 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.080816 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9483cf16-7c69-4ace-9ba7-087053569a6a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.080993 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.080895 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9483cf16-7c69-4ace-9ba7-087053569a6a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.080993 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.080919 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r67rk\" (UniqueName: \"kubernetes.io/projected/9483cf16-7c69-4ace-9ba7-087053569a6a-kube-api-access-r67rk\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.080993 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.080952 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9483cf16-7c69-4ace-9ba7-087053569a6a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.112893 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.112866 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:51.182336 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.182006 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9483cf16-7c69-4ace-9ba7-087053569a6a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.182336 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.182041 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r67rk\" (UniqueName: \"kubernetes.io/projected/9483cf16-7c69-4ace-9ba7-087053569a6a-kube-api-access-r67rk\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.182336 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.182063 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9483cf16-7c69-4ace-9ba7-087053569a6a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.182336 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.182156 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9483cf16-7c69-4ace-9ba7-087053569a6a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.182336 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.182209 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9483cf16-7c69-4ace-9ba7-087053569a6a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.182336 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.182266 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9483cf16-7c69-4ace-9ba7-087053569a6a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.182336 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.182311 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9483cf16-7c69-4ace-9ba7-087053569a6a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.183162 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.183086 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9483cf16-7c69-4ace-9ba7-087053569a6a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.184893 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.184847 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9483cf16-7c69-4ace-9ba7-087053569a6a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.185084 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.185066 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9483cf16-7c69-4ace-9ba7-087053569a6a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.185158 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.185106 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9483cf16-7c69-4ace-9ba7-087053569a6a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.185766 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.185744 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9483cf16-7c69-4ace-9ba7-087053569a6a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.194051 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.193836 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r67rk\" (UniqueName: \"kubernetes.io/projected/9483cf16-7c69-4ace-9ba7-087053569a6a-kube-api-access-r67rk\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.194701 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.194683 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9483cf16-7c69-4ace-9ba7-087053569a6a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2rlnt\" (UID: \"9483cf16-7c69-4ace-9ba7-087053569a6a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.244572 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.244548 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764"] Jul 16 12:25:51.246362 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:25:51.246338 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f90567_cca7_491d_8c6f_a08ce954afb7.slice/crio-109ff13b4f6919dd4bc4a8c5bc58a78e27e6decd9bb2a21e9e1acd141d174554 WatchSource:0}: Error finding container 109ff13b4f6919dd4bc4a8c5bc58a78e27e6decd9bb2a21e9e1acd141d174554: Status 404 returned error can't find the container with id 109ff13b4f6919dd4bc4a8c5bc58a78e27e6decd9bb2a21e9e1acd141d174554 Jul 16 12:25:51.248140 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.248119 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 12:25:51.271722 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.271698 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:25:51.405984 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.405959 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt"] Jul 16 12:25:51.408188 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:25:51.408164 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9483cf16_7c69_4ace_9ba7_087053569a6a.slice/crio-138e2fb2d3547941ba8deb6b0aeaeea77d2dd16f27a2cec766501ae3269e1649 WatchSource:0}: Error finding container 138e2fb2d3547941ba8deb6b0aeaeea77d2dd16f27a2cec766501ae3269e1649: Status 404 returned error can't find the container with id 138e2fb2d3547941ba8deb6b0aeaeea77d2dd16f27a2cec766501ae3269e1649 Jul 16 12:25:51.913935 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.913904 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" event={"ID":"9483cf16-7c69-4ace-9ba7-087053569a6a","Type":"ContainerStarted","Data":"138e2fb2d3547941ba8deb6b0aeaeea77d2dd16f27a2cec766501ae3269e1649"} Jul 16 12:25:51.915281 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:51.915249 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" event={"ID":"44f90567-cca7-491d-8c6f-a08ce954afb7","Type":"ContainerStarted","Data":"109ff13b4f6919dd4bc4a8c5bc58a78e27e6decd9bb2a21e9e1acd141d174554"} Jul 16 12:25:53.729991 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:53.729962 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:25:53.730233 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:53.730028 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:25:53.730233 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:53.730054 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:25:53.926883 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:53.926849 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" event={"ID":"44f90567-cca7-491d-8c6f-a08ce954afb7","Type":"ContainerStarted","Data":"dda557008df906a07ae14eaa883ae7e085ce22caaab1d859dcf6c852540be360"} Jul 16 12:25:53.951631 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:53.951564 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" podStartSLOduration=1.470078285 podStartE2EDuration="3.951546639s" totalImagesPullingTime="2.481468354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:25:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:25:53.951248383 +0000 UTC m=+317.813149628" watchObservedRunningTime="2026-07-16 12:25:53.951546639 +0000 UTC m=+317.813447886" Jul 16 12:25:54.113502 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:54.113429 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:54.118432 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:54.118409 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:54.931214 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:54.931183 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:25:54.932051 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:25:54.932033 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678nl764" Jul 16 12:26:03.764192 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:03.764149 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:26:03.764511 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:03.764258 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:26:03.971634 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:03.971597 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" event={"ID":"9483cf16-7c69-4ace-9ba7-087053569a6a","Type":"ContainerStarted","Data":"4169b3f3de37ec55e22d416a81b3b5994485fe2649117ad21e2d8a049c624045"} Jul 16 12:26:03.971806 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:03.971749 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:26:03.997082 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:03.997022 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" podStartSLOduration=1.643342132 podStartE2EDuration="13.997001527s" totalImagesPullingTime="12.353659395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:25:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:26:03.993777804 +0000 UTC m=+327.855679050" watchObservedRunningTime="2026-07-16 12:26:03.997001527 +0000 UTC m=+327.858902773" Jul 16 12:26:04.978138 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:04.978108 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2rlnt" Jul 16 12:26:59.604668 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.604634 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj"] Jul 16 12:26:59.607969 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.607954 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" Jul 16 12:26:59.610123 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.610096 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 12:26:59.610244 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.610163 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 12:26:59.610244 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.610179 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jrqsj\"" Jul 16 12:26:59.616401 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.616376 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj"] Jul 16 12:26:59.753645 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.753610 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/358e5b93-c06a-4028-978a-3b257fbae74f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj\" (UID: \"358e5b93-c06a-4028-978a-3b257fbae74f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" Jul 16 12:26:59.753844 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.753661 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mznq\" (UniqueName: \"kubernetes.io/projected/358e5b93-c06a-4028-978a-3b257fbae74f-kube-api-access-8mznq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj\" (UID: \"358e5b93-c06a-4028-978a-3b257fbae74f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" Jul 16 12:26:59.753844 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.753786 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/358e5b93-c06a-4028-978a-3b257fbae74f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj\" (UID: \"358e5b93-c06a-4028-978a-3b257fbae74f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" Jul 16 12:26:59.854919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.854858 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/358e5b93-c06a-4028-978a-3b257fbae74f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj\" (UID: \"358e5b93-c06a-4028-978a-3b257fbae74f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" Jul 16 12:26:59.854919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.854893 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mznq\" (UniqueName: \"kubernetes.io/projected/358e5b93-c06a-4028-978a-3b257fbae74f-kube-api-access-8mznq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj\" (UID: \"358e5b93-c06a-4028-978a-3b257fbae74f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" Jul 16 12:26:59.854919 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.854914 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/358e5b93-c06a-4028-978a-3b257fbae74f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj\" (UID: \"358e5b93-c06a-4028-978a-3b257fbae74f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" Jul 16 12:26:59.855219 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.855203 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/358e5b93-c06a-4028-978a-3b257fbae74f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj\" (UID: \"358e5b93-c06a-4028-978a-3b257fbae74f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" Jul 16 12:26:59.855256 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.855233 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/358e5b93-c06a-4028-978a-3b257fbae74f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj\" (UID: \"358e5b93-c06a-4028-978a-3b257fbae74f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" Jul 16 12:26:59.865015 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.864992 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mznq\" (UniqueName: \"kubernetes.io/projected/358e5b93-c06a-4028-978a-3b257fbae74f-kube-api-access-8mznq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj\" (UID: \"358e5b93-c06a-4028-978a-3b257fbae74f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" Jul 16 12:26:59.918239 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:26:59.918216 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" Jul 16 12:27:00.045612 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.045589 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj"] Jul 16 12:27:00.047400 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:27:00.047367 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358e5b93_c06a_4028_978a_3b257fbae74f.slice/crio-545ac54edec15bcf9391cede20cd02fc98b29a76521059a75dd052df9471a2e1 WatchSource:0}: Error finding container 545ac54edec15bcf9391cede20cd02fc98b29a76521059a75dd052df9471a2e1: Status 404 returned error can't find the container with id 545ac54edec15bcf9391cede20cd02fc98b29a76521059a75dd052df9471a2e1 Jul 16 12:27:00.200420 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.200392 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg"] Jul 16 12:27:00.203544 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.203530 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" Jul 16 12:27:00.211685 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.211663 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg"] Jul 16 12:27:00.216192 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.216172 2577 generic.go:356] "Generic (PLEG): container finished" podID="358e5b93-c06a-4028-978a-3b257fbae74f" containerID="0d6e04ddbeee7a040de36833e20034fb38699c6130a6f1ae188f09d45c3d018d" exitCode=0 Jul 16 12:27:00.216321 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.216265 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" event={"ID":"358e5b93-c06a-4028-978a-3b257fbae74f","Type":"ContainerDied","Data":"0d6e04ddbeee7a040de36833e20034fb38699c6130a6f1ae188f09d45c3d018d"} Jul 16 12:27:00.216370 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.216342 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" event={"ID":"358e5b93-c06a-4028-978a-3b257fbae74f","Type":"ContainerStarted","Data":"545ac54edec15bcf9391cede20cd02fc98b29a76521059a75dd052df9471a2e1"} Jul 16 12:27:00.359011 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.358980 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg\" (UID: \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" Jul 16 12:27:00.359176 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.359026 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg\" (UID: \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" Jul 16 12:27:00.359176 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.359048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc5hr\" (UniqueName: \"kubernetes.io/projected/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-kube-api-access-gc5hr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg\" (UID: \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" Jul 16 12:27:00.459521 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.459489 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg\" (UID: \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" Jul 16 12:27:00.459656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.459526 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg\" (UID: \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" Jul 16 12:27:00.459656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.459548 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc5hr\" (UniqueName: \"kubernetes.io/projected/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-kube-api-access-gc5hr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg\" (UID: \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" Jul 16 12:27:00.459862 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.459845 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg\" (UID: \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" Jul 16 12:27:00.459907 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.459876 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg\" (UID: \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" Jul 16 12:27:00.468741 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.468723 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc5hr\" (UniqueName: \"kubernetes.io/projected/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-kube-api-access-gc5hr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg\" (UID: \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" Jul 16 12:27:00.512596 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.512568 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" Jul 16 12:27:00.641286 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.641264 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg"] Jul 16 12:27:00.642946 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:27:00.642924 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd32bd2_ecd8_4b0a_8ec9_d473c3000dce.slice/crio-9ef801684df9a3739f5b8d8e5eabb0ba71b1fec5f43993c4ded32c6cad5fe5e4 WatchSource:0}: Error finding container 9ef801684df9a3739f5b8d8e5eabb0ba71b1fec5f43993c4ded32c6cad5fe5e4: Status 404 returned error can't find the container with id 9ef801684df9a3739f5b8d8e5eabb0ba71b1fec5f43993c4ded32c6cad5fe5e4 Jul 16 12:27:00.804612 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.804577 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb"] Jul 16 12:27:00.809846 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.809818 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" Jul 16 12:27:00.814778 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.814755 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb"] Jul 16 12:27:00.963858 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.963827 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/041982e5-e1fd-45f3-a105-3a794ef52d18-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb\" (UID: \"041982e5-e1fd-45f3-a105-3a794ef52d18\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" Jul 16 12:27:00.964009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.963873 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxrv6\" (UniqueName: \"kubernetes.io/projected/041982e5-e1fd-45f3-a105-3a794ef52d18-kube-api-access-nxrv6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb\" (UID: \"041982e5-e1fd-45f3-a105-3a794ef52d18\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" Jul 16 12:27:00.964009 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:00.963930 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/041982e5-e1fd-45f3-a105-3a794ef52d18-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb\" (UID: \"041982e5-e1fd-45f3-a105-3a794ef52d18\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" Jul 16 12:27:01.065004 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.064978 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/041982e5-e1fd-45f3-a105-3a794ef52d18-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb\" (UID: \"041982e5-e1fd-45f3-a105-3a794ef52d18\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" Jul 16 12:27:01.065094 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.065030 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxrv6\" (UniqueName: \"kubernetes.io/projected/041982e5-e1fd-45f3-a105-3a794ef52d18-kube-api-access-nxrv6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb\" (UID: \"041982e5-e1fd-45f3-a105-3a794ef52d18\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" Jul 16 12:27:01.065094 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.065085 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/041982e5-e1fd-45f3-a105-3a794ef52d18-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb\" (UID: \"041982e5-e1fd-45f3-a105-3a794ef52d18\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" Jul 16 12:27:01.065348 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.065329 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/041982e5-e1fd-45f3-a105-3a794ef52d18-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb\" (UID: \"041982e5-e1fd-45f3-a105-3a794ef52d18\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" Jul 16 12:27:01.065477 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.065459 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/041982e5-e1fd-45f3-a105-3a794ef52d18-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb\" (UID: \"041982e5-e1fd-45f3-a105-3a794ef52d18\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" Jul 16 12:27:01.073170 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.073148 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxrv6\" (UniqueName: \"kubernetes.io/projected/041982e5-e1fd-45f3-a105-3a794ef52d18-kube-api-access-nxrv6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb\" (UID: \"041982e5-e1fd-45f3-a105-3a794ef52d18\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" Jul 16 12:27:01.120067 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.120048 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" Jul 16 12:27:01.206383 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.206351 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj"] Jul 16 12:27:01.210603 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.210580 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" Jul 16 12:27:01.218703 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.218472 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj"] Jul 16 12:27:01.224836 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.224808 2577 generic.go:356] "Generic (PLEG): container finished" podID="358e5b93-c06a-4028-978a-3b257fbae74f" containerID="16dcc4b6c5751cc83a986e3971f9642df78f1291e27130f1a068385a39261337" exitCode=0 Jul 16 12:27:01.224949 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.224900 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" event={"ID":"358e5b93-c06a-4028-978a-3b257fbae74f","Type":"ContainerDied","Data":"16dcc4b6c5751cc83a986e3971f9642df78f1291e27130f1a068385a39261337"} Jul 16 12:27:01.226734 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.226708 2577 generic.go:356] "Generic (PLEG): container finished" podID="bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce" containerID="725f96ff2814ebb9428b56580311932c2e3b77e4c875dcde9ca1d68b85df375d" exitCode=0 Jul 16 12:27:01.226880 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.226761 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" event={"ID":"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce","Type":"ContainerDied","Data":"725f96ff2814ebb9428b56580311932c2e3b77e4c875dcde9ca1d68b85df375d"} Jul 16 12:27:01.226880 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.226789 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" event={"ID":"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce","Type":"ContainerStarted","Data":"9ef801684df9a3739f5b8d8e5eabb0ba71b1fec5f43993c4ded32c6cad5fe5e4"} Jul 16 12:27:01.257236 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.257215 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb"] Jul 16 12:27:01.259752 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:27:01.259729 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod041982e5_e1fd_45f3_a105_3a794ef52d18.slice/crio-dcdb7838182c3582a2e5e9478b8547da4419b46784b5cfbcd0da4db8ba6e4ad1 WatchSource:0}: Error finding container dcdb7838182c3582a2e5e9478b8547da4419b46784b5cfbcd0da4db8ba6e4ad1: Status 404 returned error can't find the container with id dcdb7838182c3582a2e5e9478b8547da4419b46784b5cfbcd0da4db8ba6e4ad1 Jul 16 12:27:01.266769 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.266746 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njvkm\" (UniqueName: \"kubernetes.io/projected/824bb465-eeba-430d-918b-6f69632cbadf-kube-api-access-njvkm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj\" (UID: \"824bb465-eeba-430d-918b-6f69632cbadf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" Jul 16 12:27:01.266834 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.266808 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/824bb465-eeba-430d-918b-6f69632cbadf-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj\" (UID: \"824bb465-eeba-430d-918b-6f69632cbadf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" Jul 16 12:27:01.266916 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.266899 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/824bb465-eeba-430d-918b-6f69632cbadf-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj\" (UID: \"824bb465-eeba-430d-918b-6f69632cbadf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" Jul 16 12:27:01.367623 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.367597 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njvkm\" (UniqueName: \"kubernetes.io/projected/824bb465-eeba-430d-918b-6f69632cbadf-kube-api-access-njvkm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj\" (UID: \"824bb465-eeba-430d-918b-6f69632cbadf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" Jul 16 12:27:01.367732 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.367656 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/824bb465-eeba-430d-918b-6f69632cbadf-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj\" (UID: \"824bb465-eeba-430d-918b-6f69632cbadf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" Jul 16 12:27:01.367732 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.367675 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/824bb465-eeba-430d-918b-6f69632cbadf-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj\" (UID: \"824bb465-eeba-430d-918b-6f69632cbadf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" Jul 16 12:27:01.367952 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.367938 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/824bb465-eeba-430d-918b-6f69632cbadf-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj\" (UID: \"824bb465-eeba-430d-918b-6f69632cbadf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" Jul 16 12:27:01.367987 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.367976 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/824bb465-eeba-430d-918b-6f69632cbadf-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj\" (UID: \"824bb465-eeba-430d-918b-6f69632cbadf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" Jul 16 12:27:01.375845 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.375828 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njvkm\" (UniqueName: \"kubernetes.io/projected/824bb465-eeba-430d-918b-6f69632cbadf-kube-api-access-njvkm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj\" (UID: \"824bb465-eeba-430d-918b-6f69632cbadf\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" Jul 16 12:27:01.523000 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.522963 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" Jul 16 12:27:01.652749 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:01.652728 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj"] Jul 16 12:27:01.654766 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:27:01.654744 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod824bb465_eeba_430d_918b_6f69632cbadf.slice/crio-45c1de3caa20c453030345b726cc96e567d92c63bb75a0793ad4b18563c767c9 WatchSource:0}: Error finding container 45c1de3caa20c453030345b726cc96e567d92c63bb75a0793ad4b18563c767c9: Status 404 returned error can't find the container with id 45c1de3caa20c453030345b726cc96e567d92c63bb75a0793ad4b18563c767c9 Jul 16 12:27:02.234074 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:02.234040 2577 generic.go:356] "Generic (PLEG): container finished" podID="824bb465-eeba-430d-918b-6f69632cbadf" containerID="36d31755d8be435071d3ee4a1649da7d7c3f469a5f29244e4911c243fa68461f" exitCode=0 Jul 16 12:27:02.234259 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:02.234135 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" event={"ID":"824bb465-eeba-430d-918b-6f69632cbadf","Type":"ContainerDied","Data":"36d31755d8be435071d3ee4a1649da7d7c3f469a5f29244e4911c243fa68461f"} Jul 16 12:27:02.234259 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:02.234195 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" event={"ID":"824bb465-eeba-430d-918b-6f69632cbadf","Type":"ContainerStarted","Data":"45c1de3caa20c453030345b726cc96e567d92c63bb75a0793ad4b18563c767c9"} Jul 16 12:27:02.236058 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:02.236033 2577 generic.go:356] "Generic (PLEG): container finished" podID="041982e5-e1fd-45f3-a105-3a794ef52d18" containerID="7d2f88b9a74ce37855ccc4e0b891957d648372aec6585122ba13a67cbd2ec9a6" exitCode=0 Jul 16 12:27:02.236159 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:02.236122 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" event={"ID":"041982e5-e1fd-45f3-a105-3a794ef52d18","Type":"ContainerDied","Data":"7d2f88b9a74ce37855ccc4e0b891957d648372aec6585122ba13a67cbd2ec9a6"} Jul 16 12:27:02.236222 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:02.236169 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" event={"ID":"041982e5-e1fd-45f3-a105-3a794ef52d18","Type":"ContainerStarted","Data":"dcdb7838182c3582a2e5e9478b8547da4419b46784b5cfbcd0da4db8ba6e4ad1"} Jul 16 12:27:02.238719 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:02.238700 2577 generic.go:356] "Generic (PLEG): container finished" podID="358e5b93-c06a-4028-978a-3b257fbae74f" containerID="49768e38f391129b6b2164accc0c38eaa01660f04be0c5a382a1bda229783f15" exitCode=0 Jul 16 12:27:02.238818 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:02.238729 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" event={"ID":"358e5b93-c06a-4028-978a-3b257fbae74f","Type":"ContainerDied","Data":"49768e38f391129b6b2164accc0c38eaa01660f04be0c5a382a1bda229783f15"} Jul 16 12:27:03.245097 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.245066 2577 generic.go:356] "Generic (PLEG): container finished" podID="041982e5-e1fd-45f3-a105-3a794ef52d18" containerID="1f5b9f769cbfddeb7a4ce3c7c7485a6e88c6f66309dbc1629e8e26fa65429885" exitCode=0 Jul 16 12:27:03.245589 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.245192 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" event={"ID":"041982e5-e1fd-45f3-a105-3a794ef52d18","Type":"ContainerDied","Data":"1f5b9f769cbfddeb7a4ce3c7c7485a6e88c6f66309dbc1629e8e26fa65429885"} Jul 16 12:27:03.247146 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.247126 2577 generic.go:356] "Generic (PLEG): container finished" podID="bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce" containerID="b810d899b138d9d2058d229223cc054dcde66336f434d39fb3196705f832ea4d" exitCode=0 Jul 16 12:27:03.247221 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.247172 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" event={"ID":"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce","Type":"ContainerDied","Data":"b810d899b138d9d2058d229223cc054dcde66336f434d39fb3196705f832ea4d"} Jul 16 12:27:03.248977 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.248954 2577 generic.go:356] "Generic (PLEG): container finished" podID="824bb465-eeba-430d-918b-6f69632cbadf" containerID="fb1b0e20fe066d18007cd314e9bbc227146f71d5b5e5a1886f3fb2045134d8f1" exitCode=0 Jul 16 12:27:03.249058 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.249023 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" event={"ID":"824bb465-eeba-430d-918b-6f69632cbadf","Type":"ContainerDied","Data":"fb1b0e20fe066d18007cd314e9bbc227146f71d5b5e5a1886f3fb2045134d8f1"} Jul 16 12:27:03.379055 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.378976 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" Jul 16 12:27:03.383472 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.383447 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/358e5b93-c06a-4028-978a-3b257fbae74f-bundle\") pod \"358e5b93-c06a-4028-978a-3b257fbae74f\" (UID: \"358e5b93-c06a-4028-978a-3b257fbae74f\") " Jul 16 12:27:03.383594 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.383496 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mznq\" (UniqueName: \"kubernetes.io/projected/358e5b93-c06a-4028-978a-3b257fbae74f-kube-api-access-8mznq\") pod \"358e5b93-c06a-4028-978a-3b257fbae74f\" (UID: \"358e5b93-c06a-4028-978a-3b257fbae74f\") " Jul 16 12:27:03.383594 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.383530 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/358e5b93-c06a-4028-978a-3b257fbae74f-util\") pod \"358e5b93-c06a-4028-978a-3b257fbae74f\" (UID: \"358e5b93-c06a-4028-978a-3b257fbae74f\") " Jul 16 12:27:03.383941 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.383909 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358e5b93-c06a-4028-978a-3b257fbae74f-bundle" (OuterVolumeSpecName: "bundle") pod "358e5b93-c06a-4028-978a-3b257fbae74f" (UID: "358e5b93-c06a-4028-978a-3b257fbae74f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:27:03.385818 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.385795 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/358e5b93-c06a-4028-978a-3b257fbae74f-kube-api-access-8mznq" (OuterVolumeSpecName: "kube-api-access-8mznq") pod "358e5b93-c06a-4028-978a-3b257fbae74f" (UID: "358e5b93-c06a-4028-978a-3b257fbae74f"). InnerVolumeSpecName "kube-api-access-8mznq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:27:03.389316 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.389231 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358e5b93-c06a-4028-978a-3b257fbae74f-util" (OuterVolumeSpecName: "util") pod "358e5b93-c06a-4028-978a-3b257fbae74f" (UID: "358e5b93-c06a-4028-978a-3b257fbae74f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:27:03.484346 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.484287 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/358e5b93-c06a-4028-978a-3b257fbae74f-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:03.484346 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.484342 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mznq\" (UniqueName: \"kubernetes.io/projected/358e5b93-c06a-4028-978a-3b257fbae74f-kube-api-access-8mznq\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:03.484346 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:03.484352 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/358e5b93-c06a-4028-978a-3b257fbae74f-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:04.256189 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:04.256158 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" event={"ID":"358e5b93-c06a-4028-978a-3b257fbae74f","Type":"ContainerDied","Data":"545ac54edec15bcf9391cede20cd02fc98b29a76521059a75dd052df9471a2e1"} Jul 16 12:27:04.256646 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:04.256211 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="545ac54edec15bcf9391cede20cd02fc98b29a76521059a75dd052df9471a2e1" Jul 16 12:27:04.256646 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:04.256166 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj" Jul 16 12:27:04.258475 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:04.258446 2577 generic.go:356] "Generic (PLEG): container finished" podID="bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce" containerID="a3d12ec82e76722c4a5d5e1639424ba661f34aea82cca92c39d2c42268c8c8fe" exitCode=0 Jul 16 12:27:04.258594 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:04.258488 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" event={"ID":"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce","Type":"ContainerDied","Data":"a3d12ec82e76722c4a5d5e1639424ba661f34aea82cca92c39d2c42268c8c8fe"} Jul 16 12:27:04.260746 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:04.260723 2577 generic.go:356] "Generic (PLEG): container finished" podID="824bb465-eeba-430d-918b-6f69632cbadf" containerID="1aed846cf768275e022b436955add01adfe1bed2c5af6ab32fc47bc4f227014e" exitCode=0 Jul 16 12:27:04.260828 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:04.260805 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" event={"ID":"824bb465-eeba-430d-918b-6f69632cbadf","Type":"ContainerDied","Data":"1aed846cf768275e022b436955add01adfe1bed2c5af6ab32fc47bc4f227014e"} Jul 16 12:27:04.262805 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:04.262785 2577 generic.go:356] "Generic (PLEG): container finished" podID="041982e5-e1fd-45f3-a105-3a794ef52d18" containerID="37e8173886b8cb835aff5a1b21ff7a12b865a7ff501a65c6bfb2552e31c63373" exitCode=0 Jul 16 12:27:04.262907 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:04.262874 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" event={"ID":"041982e5-e1fd-45f3-a105-3a794ef52d18","Type":"ContainerDied","Data":"37e8173886b8cb835aff5a1b21ff7a12b865a7ff501a65c6bfb2552e31c63373"} Jul 16 12:27:05.422197 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.422176 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" Jul 16 12:27:05.441741 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.441720 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" Jul 16 12:27:05.445455 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.445435 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" Jul 16 12:27:05.501604 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.501573 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gc5hr\" (UniqueName: \"kubernetes.io/projected/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-kube-api-access-gc5hr\") pod \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\" (UID: \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\") " Jul 16 12:27:05.501746 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.501615 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/041982e5-e1fd-45f3-a105-3a794ef52d18-util\") pod \"041982e5-e1fd-45f3-a105-3a794ef52d18\" (UID: \"041982e5-e1fd-45f3-a105-3a794ef52d18\") " Jul 16 12:27:05.501746 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.501651 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/041982e5-e1fd-45f3-a105-3a794ef52d18-bundle\") pod \"041982e5-e1fd-45f3-a105-3a794ef52d18\" (UID: \"041982e5-e1fd-45f3-a105-3a794ef52d18\") " Jul 16 12:27:05.501746 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.501696 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxrv6\" (UniqueName: \"kubernetes.io/projected/041982e5-e1fd-45f3-a105-3a794ef52d18-kube-api-access-nxrv6\") pod \"041982e5-e1fd-45f3-a105-3a794ef52d18\" (UID: \"041982e5-e1fd-45f3-a105-3a794ef52d18\") " Jul 16 12:27:05.501871 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.501746 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-bundle\") pod \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\" (UID: \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\") " Jul 16 12:27:05.501871 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.501772 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njvkm\" (UniqueName: \"kubernetes.io/projected/824bb465-eeba-430d-918b-6f69632cbadf-kube-api-access-njvkm\") pod \"824bb465-eeba-430d-918b-6f69632cbadf\" (UID: \"824bb465-eeba-430d-918b-6f69632cbadf\") " Jul 16 12:27:05.501871 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.501810 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/824bb465-eeba-430d-918b-6f69632cbadf-util\") pod \"824bb465-eeba-430d-918b-6f69632cbadf\" (UID: \"824bb465-eeba-430d-918b-6f69632cbadf\") " Jul 16 12:27:05.501871 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.501837 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-util\") pod \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\" (UID: \"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce\") " Jul 16 12:27:05.501871 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.501860 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/824bb465-eeba-430d-918b-6f69632cbadf-bundle\") pod \"824bb465-eeba-430d-918b-6f69632cbadf\" (UID: \"824bb465-eeba-430d-918b-6f69632cbadf\") " Jul 16 12:27:05.504631 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.502535 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-bundle" (OuterVolumeSpecName: "bundle") pod "bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce" (UID: "bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:27:05.504631 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.502695 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/041982e5-e1fd-45f3-a105-3a794ef52d18-bundle" (OuterVolumeSpecName: "bundle") pod "041982e5-e1fd-45f3-a105-3a794ef52d18" (UID: "041982e5-e1fd-45f3-a105-3a794ef52d18"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:27:05.504631 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.503266 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824bb465-eeba-430d-918b-6f69632cbadf-bundle" (OuterVolumeSpecName: "bundle") pod "824bb465-eeba-430d-918b-6f69632cbadf" (UID: "824bb465-eeba-430d-918b-6f69632cbadf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:27:05.504631 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.504461 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-kube-api-access-gc5hr" (OuterVolumeSpecName: "kube-api-access-gc5hr") pod "bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce" (UID: "bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce"). InnerVolumeSpecName "kube-api-access-gc5hr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:27:05.504953 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.504813 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/041982e5-e1fd-45f3-a105-3a794ef52d18-kube-api-access-nxrv6" (OuterVolumeSpecName: "kube-api-access-nxrv6") pod "041982e5-e1fd-45f3-a105-3a794ef52d18" (UID: "041982e5-e1fd-45f3-a105-3a794ef52d18"). InnerVolumeSpecName "kube-api-access-nxrv6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:27:05.505010 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.504952 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/824bb465-eeba-430d-918b-6f69632cbadf-kube-api-access-njvkm" (OuterVolumeSpecName: "kube-api-access-njvkm") pod "824bb465-eeba-430d-918b-6f69632cbadf" (UID: "824bb465-eeba-430d-918b-6f69632cbadf"). InnerVolumeSpecName "kube-api-access-njvkm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:27:05.509049 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.509029 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/041982e5-e1fd-45f3-a105-3a794ef52d18-util" (OuterVolumeSpecName: "util") pod "041982e5-e1fd-45f3-a105-3a794ef52d18" (UID: "041982e5-e1fd-45f3-a105-3a794ef52d18"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:27:05.509355 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.509329 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-util" (OuterVolumeSpecName: "util") pod "bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce" (UID: "bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:27:05.510482 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.510461 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824bb465-eeba-430d-918b-6f69632cbadf-util" (OuterVolumeSpecName: "util") pod "824bb465-eeba-430d-918b-6f69632cbadf" (UID: "824bb465-eeba-430d-918b-6f69632cbadf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:27:05.603406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.603347 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/824bb465-eeba-430d-918b-6f69632cbadf-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:05.603406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.603368 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc5hr\" (UniqueName: \"kubernetes.io/projected/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-kube-api-access-gc5hr\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:05.603406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.603378 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/041982e5-e1fd-45f3-a105-3a794ef52d18-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:05.603406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.603385 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/041982e5-e1fd-45f3-a105-3a794ef52d18-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:05.603406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.603393 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxrv6\" (UniqueName: \"kubernetes.io/projected/041982e5-e1fd-45f3-a105-3a794ef52d18-kube-api-access-nxrv6\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:05.603406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.603400 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:05.603406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.603409 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njvkm\" (UniqueName: \"kubernetes.io/projected/824bb465-eeba-430d-918b-6f69632cbadf-kube-api-access-njvkm\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:05.603671 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.603416 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/824bb465-eeba-430d-918b-6f69632cbadf-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:05.603671 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:05.603423 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce-util\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:06.274005 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:06.273978 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" Jul 16 12:27:06.274181 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:06.274019 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb" event={"ID":"041982e5-e1fd-45f3-a105-3a794ef52d18","Type":"ContainerDied","Data":"dcdb7838182c3582a2e5e9478b8547da4419b46784b5cfbcd0da4db8ba6e4ad1"} Jul 16 12:27:06.274181 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:06.274068 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcdb7838182c3582a2e5e9478b8547da4419b46784b5cfbcd0da4db8ba6e4ad1" Jul 16 12:27:06.275971 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:06.275949 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" Jul 16 12:27:06.275971 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:06.275954 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg" event={"ID":"bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce","Type":"ContainerDied","Data":"9ef801684df9a3739f5b8d8e5eabb0ba71b1fec5f43993c4ded32c6cad5fe5e4"} Jul 16 12:27:06.276136 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:06.275992 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ef801684df9a3739f5b8d8e5eabb0ba71b1fec5f43993c4ded32c6cad5fe5e4" Jul 16 12:27:06.277917 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:06.277896 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" event={"ID":"824bb465-eeba-430d-918b-6f69632cbadf","Type":"ContainerDied","Data":"45c1de3caa20c453030345b726cc96e567d92c63bb75a0793ad4b18563c767c9"} Jul 16 12:27:06.278022 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:06.277924 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj" Jul 16 12:27:06.278022 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:06.277935 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45c1de3caa20c453030345b726cc96e567d92c63bb75a0793ad4b18563c767c9" Jul 16 12:27:24.094778 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:24.094736 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9bdd6df8d-6m4gf"] Jul 16 12:27:49.115704 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.115642 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9bdd6df8d-6m4gf" podUID="353ebb58-92bb-4fa3-904c-3c8ab81b2e72" containerName="console" containerID="cri-o://ed1c0f34f234e8af96dc93da508c9050d5400e77a2f0dd050f7129792d92bf45" gracePeriod=15 Jul 16 12:27:49.364013 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.363990 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:27:49.464887 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.464793 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7stw\" (UniqueName: \"kubernetes.io/projected/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-kube-api-access-b7stw\") pod \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " Jul 16 12:27:49.465068 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.464937 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-service-ca\") pod \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " Jul 16 12:27:49.465068 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.464972 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-config\") pod \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " Jul 16 12:27:49.465068 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.464999 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-serving-cert\") pod \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " Jul 16 12:27:49.465238 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.465082 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-oauth-config\") pod \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " Jul 16 12:27:49.465238 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.465153 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-oauth-serving-cert\") pod \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " Jul 16 12:27:49.465238 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.465185 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-trusted-ca-bundle\") pod \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\" (UID: \"353ebb58-92bb-4fa3-904c-3c8ab81b2e72\") " Jul 16 12:27:49.465436 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.465381 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-service-ca" (OuterVolumeSpecName: "service-ca") pod "353ebb58-92bb-4fa3-904c-3c8ab81b2e72" (UID: "353ebb58-92bb-4fa3-904c-3c8ab81b2e72"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:27:49.465436 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.465407 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-config" (OuterVolumeSpecName: "console-config") pod "353ebb58-92bb-4fa3-904c-3c8ab81b2e72" (UID: "353ebb58-92bb-4fa3-904c-3c8ab81b2e72"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:27:49.465774 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.465613 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-service-ca\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:49.465774 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.465639 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-config\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:49.465774 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.465615 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "353ebb58-92bb-4fa3-904c-3c8ab81b2e72" (UID: "353ebb58-92bb-4fa3-904c-3c8ab81b2e72"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:27:49.465774 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.465697 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "353ebb58-92bb-4fa3-904c-3c8ab81b2e72" (UID: "353ebb58-92bb-4fa3-904c-3c8ab81b2e72"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:27:49.467002 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.466981 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-kube-api-access-b7stw" (OuterVolumeSpecName: "kube-api-access-b7stw") pod "353ebb58-92bb-4fa3-904c-3c8ab81b2e72" (UID: "353ebb58-92bb-4fa3-904c-3c8ab81b2e72"). InnerVolumeSpecName "kube-api-access-b7stw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:27:49.467284 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.467265 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "353ebb58-92bb-4fa3-904c-3c8ab81b2e72" (UID: "353ebb58-92bb-4fa3-904c-3c8ab81b2e72"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:27:49.467487 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.467468 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "353ebb58-92bb-4fa3-904c-3c8ab81b2e72" (UID: "353ebb58-92bb-4fa3-904c-3c8ab81b2e72"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:27:49.478021 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.478000 2577 generic.go:356] "Generic (PLEG): container finished" podID="353ebb58-92bb-4fa3-904c-3c8ab81b2e72" containerID="ed1c0f34f234e8af96dc93da508c9050d5400e77a2f0dd050f7129792d92bf45" exitCode=0 Jul 16 12:27:49.478111 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.478042 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9bdd6df8d-6m4gf" event={"ID":"353ebb58-92bb-4fa3-904c-3c8ab81b2e72","Type":"ContainerDied","Data":"ed1c0f34f234e8af96dc93da508c9050d5400e77a2f0dd050f7129792d92bf45"} Jul 16 12:27:49.478111 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.478078 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9bdd6df8d-6m4gf" event={"ID":"353ebb58-92bb-4fa3-904c-3c8ab81b2e72","Type":"ContainerDied","Data":"1c02076f3e73043e18648f8445a600ebb0bb3365ffce2c8ce6a43fc6a71e64de"} Jul 16 12:27:49.478111 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.478097 2577 scope.go:117] "RemoveContainer" containerID="ed1c0f34f234e8af96dc93da508c9050d5400e77a2f0dd050f7129792d92bf45" Jul 16 12:27:49.478222 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.478045 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9bdd6df8d-6m4gf" Jul 16 12:27:49.492491 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.492471 2577 scope.go:117] "RemoveContainer" containerID="ed1c0f34f234e8af96dc93da508c9050d5400e77a2f0dd050f7129792d92bf45" Jul 16 12:27:49.492782 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:27:49.492762 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed1c0f34f234e8af96dc93da508c9050d5400e77a2f0dd050f7129792d92bf45\": container with ID starting with ed1c0f34f234e8af96dc93da508c9050d5400e77a2f0dd050f7129792d92bf45 not found: ID does not exist" containerID="ed1c0f34f234e8af96dc93da508c9050d5400e77a2f0dd050f7129792d92bf45" Jul 16 12:27:49.492852 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.492792 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed1c0f34f234e8af96dc93da508c9050d5400e77a2f0dd050f7129792d92bf45"} err="failed to get container status \"ed1c0f34f234e8af96dc93da508c9050d5400e77a2f0dd050f7129792d92bf45\": rpc error: code = NotFound desc = could not find container \"ed1c0f34f234e8af96dc93da508c9050d5400e77a2f0dd050f7129792d92bf45\": container with ID starting with ed1c0f34f234e8af96dc93da508c9050d5400e77a2f0dd050f7129792d92bf45 not found: ID does not exist" Jul 16 12:27:49.503303 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.503271 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9bdd6df8d-6m4gf"] Jul 16 12:27:49.507681 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.507662 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9bdd6df8d-6m4gf"] Jul 16 12:27:49.566929 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.566900 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7stw\" (UniqueName: \"kubernetes.io/projected/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-kube-api-access-b7stw\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:49.566929 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.566926 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-serving-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:49.567052 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.566940 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-console-oauth-config\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:49.567052 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.566952 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-oauth-serving-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:49.567052 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:49.566964 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353ebb58-92bb-4fa3-904c-3c8ab81b2e72-trusted-ca-bundle\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:27:50.639034 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:27:50.638999 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="353ebb58-92bb-4fa3-904c-3c8ab81b2e72" path="/var/lib/kubelet/pods/353ebb58-92bb-4fa3-904c-3c8ab81b2e72/volumes" Jul 16 12:28:07.530941 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.530859 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn"] Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531132 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="041982e5-e1fd-45f3-a105-3a794ef52d18" containerName="extract" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531143 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="041982e5-e1fd-45f3-a105-3a794ef52d18" containerName="extract" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531152 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="353ebb58-92bb-4fa3-904c-3c8ab81b2e72" containerName="console" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531158 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="353ebb58-92bb-4fa3-904c-3c8ab81b2e72" containerName="console" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531165 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="358e5b93-c06a-4028-978a-3b257fbae74f" containerName="util" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531170 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="358e5b93-c06a-4028-978a-3b257fbae74f" containerName="util" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531175 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="041982e5-e1fd-45f3-a105-3a794ef52d18" containerName="util" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531180 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="041982e5-e1fd-45f3-a105-3a794ef52d18" containerName="util" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531187 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="824bb465-eeba-430d-918b-6f69632cbadf" containerName="util" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531191 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="824bb465-eeba-430d-918b-6f69632cbadf" containerName="util" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531198 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce" containerName="extract" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531203 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce" containerName="extract" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531209 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="358e5b93-c06a-4028-978a-3b257fbae74f" containerName="pull" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531213 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="358e5b93-c06a-4028-978a-3b257fbae74f" containerName="pull" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531220 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="824bb465-eeba-430d-918b-6f69632cbadf" containerName="pull" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531224 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="824bb465-eeba-430d-918b-6f69632cbadf" containerName="pull" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531231 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="041982e5-e1fd-45f3-a105-3a794ef52d18" containerName="pull" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531236 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="041982e5-e1fd-45f3-a105-3a794ef52d18" containerName="pull" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531241 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce" containerName="pull" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531246 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce" containerName="pull" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531251 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="824bb465-eeba-430d-918b-6f69632cbadf" containerName="extract" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531256 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="824bb465-eeba-430d-918b-6f69632cbadf" containerName="extract" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531263 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="358e5b93-c06a-4028-978a-3b257fbae74f" containerName="extract" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531268 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="358e5b93-c06a-4028-978a-3b257fbae74f" containerName="extract" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531273 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce" containerName="util" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531278 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce" containerName="util" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531356 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="353ebb58-92bb-4fa3-904c-3c8ab81b2e72" containerName="console" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531366 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce" containerName="extract" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531372 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="358e5b93-c06a-4028-978a-3b257fbae74f" containerName="extract" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531378 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="041982e5-e1fd-45f3-a105-3a794ef52d18" containerName="extract" Jul 16 12:28:07.531395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.531384 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="824bb465-eeba-430d-918b-6f69632cbadf" containerName="extract" Jul 16 12:28:07.534220 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.534204 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.536503 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.536479 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-h5h74\"" Jul 16 12:28:07.548678 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.548659 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn"] Jul 16 12:28:07.603232 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.603202 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.603232 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.603234 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.603445 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.603257 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.603445 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.603282 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvnql\" (UniqueName: \"kubernetes.io/projected/46882479-a6f2-48fe-8886-a81bbf1d101e-kube-api-access-bvnql\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.603445 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.603372 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46882479-a6f2-48fe-8886-a81bbf1d101e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.603445 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.603422 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.603585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.603456 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46882479-a6f2-48fe-8886-a81bbf1d101e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.603585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.603477 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46882479-a6f2-48fe-8886-a81bbf1d101e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.603585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.603494 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.703938 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.703907 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46882479-a6f2-48fe-8886-a81bbf1d101e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.703938 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.703944 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.704178 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.703965 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.704178 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.703982 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.704178 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.704009 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.704178 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.704042 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvnql\" (UniqueName: \"kubernetes.io/projected/46882479-a6f2-48fe-8886-a81bbf1d101e-kube-api-access-bvnql\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.704178 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.704078 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46882479-a6f2-48fe-8886-a81bbf1d101e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.704178 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.704121 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.704178 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.704150 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46882479-a6f2-48fe-8886-a81bbf1d101e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.704576 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.704489 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.704576 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.704522 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.704754 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.704600 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.704754 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.704652 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.704948 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.704820 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46882479-a6f2-48fe-8886-a81bbf1d101e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.706405 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.706382 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46882479-a6f2-48fe-8886-a81bbf1d101e-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.706550 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.706534 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46882479-a6f2-48fe-8886-a81bbf1d101e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.712204 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.712186 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46882479-a6f2-48fe-8886-a81bbf1d101e-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.712618 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.712575 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvnql\" (UniqueName: \"kubernetes.io/projected/46882479-a6f2-48fe-8886-a81bbf1d101e-kube-api-access-bvnql\") pod \"maas-default-gateway-openshift-default-5896d4dd49-fh8dn\" (UID: \"46882479-a6f2-48fe-8886-a81bbf1d101e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.844573 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.844495 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:07.996578 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:07.996554 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn"] Jul 16 12:28:07.997976 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:28:07.997948 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46882479_a6f2_48fe_8886_a81bbf1d101e.slice/crio-42ce2d5b928d566887b7187fcc5a93a1a00e72e2322622ece5b99270a2bdcb8b WatchSource:0}: Error finding container 42ce2d5b928d566887b7187fcc5a93a1a00e72e2322622ece5b99270a2bdcb8b: Status 404 returned error can't find the container with id 42ce2d5b928d566887b7187fcc5a93a1a00e72e2322622ece5b99270a2bdcb8b Jul 16 12:28:08.004841 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:08.003366 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:28:08.004841 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:08.003540 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:28:08.004841 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:08.003628 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:28:08.573635 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:08.573598 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" event={"ID":"46882479-a6f2-48fe-8886-a81bbf1d101e","Type":"ContainerStarted","Data":"fa20ec0e09548d8c7f4783183c3ba5292edf56c5ec2702100023ffe51099ad65"} Jul 16 12:28:08.574024 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:08.573687 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" event={"ID":"46882479-a6f2-48fe-8886-a81bbf1d101e","Type":"ContainerStarted","Data":"42ce2d5b928d566887b7187fcc5a93a1a00e72e2322622ece5b99270a2bdcb8b"} Jul 16 12:28:08.609877 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:08.609832 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" podStartSLOduration=1.6098180850000001 podStartE2EDuration="1.609818085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:28:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:28:08.607899272 +0000 UTC m=+452.469800514" watchObservedRunningTime="2026-07-16 12:28:08.609818085 +0000 UTC m=+452.471719330" Jul 16 12:28:08.845489 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:08.845410 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:09.849626 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:09.849595 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:10.583140 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:10.583113 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:10.584024 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:10.584005 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:28:51.663427 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.663378 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d45c57465-4cn4t"] Jul 16 12:28:51.668014 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.667994 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d45c57465-4cn4t" Jul 16 12:28:51.670253 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.670234 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 12:28:51.670422 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.670275 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bhcp6\"" Jul 16 12:28:51.671041 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.671028 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 12:28:51.683751 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.683730 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d45c57465-4cn4t"] Jul 16 12:28:51.758592 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.758564 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a423292-136a-403f-8df3-fcef9bc1e7b9-webhook-cert\") pod \"maas-controller-7d45c57465-4cn4t\" (UID: \"5a423292-136a-403f-8df3-fcef9bc1e7b9\") " pod="opendatahub/maas-controller-7d45c57465-4cn4t" Jul 16 12:28:51.758765 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.758614 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prpr4\" (UniqueName: \"kubernetes.io/projected/5a423292-136a-403f-8df3-fcef9bc1e7b9-kube-api-access-prpr4\") pod \"maas-controller-7d45c57465-4cn4t\" (UID: \"5a423292-136a-403f-8df3-fcef9bc1e7b9\") " pod="opendatahub/maas-controller-7d45c57465-4cn4t" Jul 16 12:28:51.859808 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.859768 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a423292-136a-403f-8df3-fcef9bc1e7b9-webhook-cert\") pod \"maas-controller-7d45c57465-4cn4t\" (UID: \"5a423292-136a-403f-8df3-fcef9bc1e7b9\") " pod="opendatahub/maas-controller-7d45c57465-4cn4t" Jul 16 12:28:51.859995 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.859836 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prpr4\" (UniqueName: \"kubernetes.io/projected/5a423292-136a-403f-8df3-fcef9bc1e7b9-kube-api-access-prpr4\") pod \"maas-controller-7d45c57465-4cn4t\" (UID: \"5a423292-136a-403f-8df3-fcef9bc1e7b9\") " pod="opendatahub/maas-controller-7d45c57465-4cn4t" Jul 16 12:28:51.859995 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:28:51.859928 2577 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 16 12:28:51.859995 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:28:51.859995 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a423292-136a-403f-8df3-fcef9bc1e7b9-webhook-cert podName:5a423292-136a-403f-8df3-fcef9bc1e7b9 nodeName:}" failed. No retries permitted until 2026-07-16 12:28:52.359978134 +0000 UTC m=+496.221879357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/5a423292-136a-403f-8df3-fcef9bc1e7b9-webhook-cert") pod "maas-controller-7d45c57465-4cn4t" (UID: "5a423292-136a-403f-8df3-fcef9bc1e7b9") : secret "maas-controller-webhook-cert" not found Jul 16 12:28:51.875221 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.875189 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74b78db6b9-zqz6s"] Jul 16 12:28:51.876839 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.876822 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prpr4\" (UniqueName: \"kubernetes.io/projected/5a423292-136a-403f-8df3-fcef9bc1e7b9-kube-api-access-prpr4\") pod \"maas-controller-7d45c57465-4cn4t\" (UID: \"5a423292-136a-403f-8df3-fcef9bc1e7b9\") " pod="opendatahub/maas-controller-7d45c57465-4cn4t" Jul 16 12:28:51.878521 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.878504 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b78db6b9-zqz6s" Jul 16 12:28:51.891217 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.891192 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74b78db6b9-zqz6s"] Jul 16 12:28:51.960390 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.960312 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzdrr\" (UniqueName: \"kubernetes.io/projected/083698c4-8478-4fb5-9653-365d4d8a5a4e-kube-api-access-tzdrr\") pod \"maas-controller-74b78db6b9-zqz6s\" (UID: \"083698c4-8478-4fb5-9653-365d4d8a5a4e\") " pod="opendatahub/maas-controller-74b78db6b9-zqz6s" Jul 16 12:28:51.960390 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:51.960373 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/083698c4-8478-4fb5-9653-365d4d8a5a4e-webhook-cert\") pod \"maas-controller-74b78db6b9-zqz6s\" (UID: \"083698c4-8478-4fb5-9653-365d4d8a5a4e\") " pod="opendatahub/maas-controller-74b78db6b9-zqz6s" Jul 16 12:28:52.061725 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:52.061690 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzdrr\" (UniqueName: \"kubernetes.io/projected/083698c4-8478-4fb5-9653-365d4d8a5a4e-kube-api-access-tzdrr\") pod \"maas-controller-74b78db6b9-zqz6s\" (UID: \"083698c4-8478-4fb5-9653-365d4d8a5a4e\") " pod="opendatahub/maas-controller-74b78db6b9-zqz6s" Jul 16 12:28:52.061879 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:52.061733 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/083698c4-8478-4fb5-9653-365d4d8a5a4e-webhook-cert\") pod \"maas-controller-74b78db6b9-zqz6s\" (UID: \"083698c4-8478-4fb5-9653-365d4d8a5a4e\") " pod="opendatahub/maas-controller-74b78db6b9-zqz6s" Jul 16 12:28:52.064156 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:52.064133 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/083698c4-8478-4fb5-9653-365d4d8a5a4e-webhook-cert\") pod \"maas-controller-74b78db6b9-zqz6s\" (UID: \"083698c4-8478-4fb5-9653-365d4d8a5a4e\") " pod="opendatahub/maas-controller-74b78db6b9-zqz6s" Jul 16 12:28:52.070782 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:52.070765 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzdrr\" (UniqueName: \"kubernetes.io/projected/083698c4-8478-4fb5-9653-365d4d8a5a4e-kube-api-access-tzdrr\") pod \"maas-controller-74b78db6b9-zqz6s\" (UID: \"083698c4-8478-4fb5-9653-365d4d8a5a4e\") " pod="opendatahub/maas-controller-74b78db6b9-zqz6s" Jul 16 12:28:52.193171 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:52.193131 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b78db6b9-zqz6s" Jul 16 12:28:52.321008 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:52.320983 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74b78db6b9-zqz6s"] Jul 16 12:28:52.323331 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:28:52.323287 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod083698c4_8478_4fb5_9653_365d4d8a5a4e.slice/crio-58f39f59803b345acdb0d051898eb24a2b3bc767986bb091a3c7b615a63349fd WatchSource:0}: Error finding container 58f39f59803b345acdb0d051898eb24a2b3bc767986bb091a3c7b615a63349fd: Status 404 returned error can't find the container with id 58f39f59803b345acdb0d051898eb24a2b3bc767986bb091a3c7b615a63349fd Jul 16 12:28:52.365376 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:52.365339 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a423292-136a-403f-8df3-fcef9bc1e7b9-webhook-cert\") pod \"maas-controller-7d45c57465-4cn4t\" (UID: \"5a423292-136a-403f-8df3-fcef9bc1e7b9\") " pod="opendatahub/maas-controller-7d45c57465-4cn4t" Jul 16 12:28:52.367821 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:52.367804 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a423292-136a-403f-8df3-fcef9bc1e7b9-webhook-cert\") pod \"maas-controller-7d45c57465-4cn4t\" (UID: \"5a423292-136a-403f-8df3-fcef9bc1e7b9\") " pod="opendatahub/maas-controller-7d45c57465-4cn4t" Jul 16 12:28:52.577622 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:52.577542 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d45c57465-4cn4t" Jul 16 12:28:52.707091 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:52.707067 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d45c57465-4cn4t"] Jul 16 12:28:52.708590 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:28:52.708563 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a423292_136a_403f_8df3_fcef9bc1e7b9.slice/crio-daaf64e1fcd7d189db7ed7f1e630868502e40086b909eb4c71f21e853c89af68 WatchSource:0}: Error finding container daaf64e1fcd7d189db7ed7f1e630868502e40086b909eb4c71f21e853c89af68: Status 404 returned error can't find the container with id daaf64e1fcd7d189db7ed7f1e630868502e40086b909eb4c71f21e853c89af68 Jul 16 12:28:52.781283 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:52.781243 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d45c57465-4cn4t" event={"ID":"5a423292-136a-403f-8df3-fcef9bc1e7b9","Type":"ContainerStarted","Data":"daaf64e1fcd7d189db7ed7f1e630868502e40086b909eb4c71f21e853c89af68"} Jul 16 12:28:52.782342 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:52.782323 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b78db6b9-zqz6s" event={"ID":"083698c4-8478-4fb5-9653-365d4d8a5a4e","Type":"ContainerStarted","Data":"58f39f59803b345acdb0d051898eb24a2b3bc767986bb091a3c7b615a63349fd"} Jul 16 12:28:56.807043 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:56.807005 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d45c57465-4cn4t" event={"ID":"5a423292-136a-403f-8df3-fcef9bc1e7b9","Type":"ContainerStarted","Data":"a269ec13d518f3055416271630391b290eb4a0461181c219b8c96d56242a68c7"} Jul 16 12:28:56.807523 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:56.807095 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d45c57465-4cn4t" Jul 16 12:28:56.808531 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:56.808500 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b78db6b9-zqz6s" event={"ID":"083698c4-8478-4fb5-9653-365d4d8a5a4e","Type":"ContainerStarted","Data":"92c88a2e25bb17d461c8e6de86a5e30a9358648b0491d258bd625698f893a4d4"} Jul 16 12:28:56.808640 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:56.808545 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74b78db6b9-zqz6s" Jul 16 12:28:56.825691 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:56.825645 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d45c57465-4cn4t" podStartSLOduration=2.607377757 podStartE2EDuration="5.825631s" totalImagesPullingTime="3.218253243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:28:56.824318758 +0000 UTC m=+500.686220021" watchObservedRunningTime="2026-07-16 12:28:56.825631 +0000 UTC m=+500.687532245" Jul 16 12:28:56.845329 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:28:56.845260 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74b78db6b9-zqz6s" podStartSLOduration=2.249303076 podStartE2EDuration="5.845247135s" totalImagesPullingTime="3.595944059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:28:56.844202042 +0000 UTC m=+500.706103289" watchObservedRunningTime="2026-07-16 12:28:56.845247135 +0000 UTC m=+500.707148380" Jul 16 12:29:07.815455 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:07.815424 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d45c57465-4cn4t" Jul 16 12:29:07.815850 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:07.815498 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74b78db6b9-zqz6s" Jul 16 12:29:07.874397 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:07.874359 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d45c57465-4cn4t"] Jul 16 12:29:07.874645 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:07.874614 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7d45c57465-4cn4t" podUID="5a423292-136a-403f-8df3-fcef9bc1e7b9" containerName="manager" containerID="cri-o://a269ec13d518f3055416271630391b290eb4a0461181c219b8c96d56242a68c7" gracePeriod=30 Jul 16 12:29:08.112756 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.112724 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d45c57465-4cn4t" Jul 16 12:29:08.203602 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.203567 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prpr4\" (UniqueName: \"kubernetes.io/projected/5a423292-136a-403f-8df3-fcef9bc1e7b9-kube-api-access-prpr4\") pod \"5a423292-136a-403f-8df3-fcef9bc1e7b9\" (UID: \"5a423292-136a-403f-8df3-fcef9bc1e7b9\") " Jul 16 12:29:08.203795 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.203625 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a423292-136a-403f-8df3-fcef9bc1e7b9-webhook-cert\") pod \"5a423292-136a-403f-8df3-fcef9bc1e7b9\" (UID: \"5a423292-136a-403f-8df3-fcef9bc1e7b9\") " Jul 16 12:29:08.206053 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.206027 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a423292-136a-403f-8df3-fcef9bc1e7b9-kube-api-access-prpr4" (OuterVolumeSpecName: "kube-api-access-prpr4") pod "5a423292-136a-403f-8df3-fcef9bc1e7b9" (UID: "5a423292-136a-403f-8df3-fcef9bc1e7b9"). InnerVolumeSpecName "kube-api-access-prpr4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:29:08.206142 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.206031 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a423292-136a-403f-8df3-fcef9bc1e7b9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5a423292-136a-403f-8df3-fcef9bc1e7b9" (UID: "5a423292-136a-403f-8df3-fcef9bc1e7b9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:29:08.304977 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.304932 2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a423292-136a-403f-8df3-fcef9bc1e7b9-webhook-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:29:08.304977 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.304973 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prpr4\" (UniqueName: \"kubernetes.io/projected/5a423292-136a-403f-8df3-fcef9bc1e7b9-kube-api-access-prpr4\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:29:08.865668 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.865638 2577 generic.go:356] "Generic (PLEG): container finished" podID="5a423292-136a-403f-8df3-fcef9bc1e7b9" containerID="a269ec13d518f3055416271630391b290eb4a0461181c219b8c96d56242a68c7" exitCode=0 Jul 16 12:29:08.866069 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.865683 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d45c57465-4cn4t" Jul 16 12:29:08.866069 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.865725 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d45c57465-4cn4t" event={"ID":"5a423292-136a-403f-8df3-fcef9bc1e7b9","Type":"ContainerDied","Data":"a269ec13d518f3055416271630391b290eb4a0461181c219b8c96d56242a68c7"} Jul 16 12:29:08.866069 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.865779 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d45c57465-4cn4t" event={"ID":"5a423292-136a-403f-8df3-fcef9bc1e7b9","Type":"ContainerDied","Data":"daaf64e1fcd7d189db7ed7f1e630868502e40086b909eb4c71f21e853c89af68"} Jul 16 12:29:08.866069 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.865801 2577 scope.go:117] "RemoveContainer" containerID="a269ec13d518f3055416271630391b290eb4a0461181c219b8c96d56242a68c7" Jul 16 12:29:08.874311 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.874280 2577 scope.go:117] "RemoveContainer" containerID="a269ec13d518f3055416271630391b290eb4a0461181c219b8c96d56242a68c7" Jul 16 12:29:08.874572 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:29:08.874556 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a269ec13d518f3055416271630391b290eb4a0461181c219b8c96d56242a68c7\": container with ID starting with a269ec13d518f3055416271630391b290eb4a0461181c219b8c96d56242a68c7 not found: ID does not exist" containerID="a269ec13d518f3055416271630391b290eb4a0461181c219b8c96d56242a68c7" Jul 16 12:29:08.874633 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.874581 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a269ec13d518f3055416271630391b290eb4a0461181c219b8c96d56242a68c7"} err="failed to get container status \"a269ec13d518f3055416271630391b290eb4a0461181c219b8c96d56242a68c7\": rpc error: code = NotFound desc = could not find container \"a269ec13d518f3055416271630391b290eb4a0461181c219b8c96d56242a68c7\": container with ID starting with a269ec13d518f3055416271630391b290eb4a0461181c219b8c96d56242a68c7 not found: ID does not exist" Jul 16 12:29:08.881967 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.881939 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d45c57465-4cn4t"] Jul 16 12:29:08.886392 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:08.886363 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d45c57465-4cn4t"] Jul 16 12:29:10.638208 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:10.638176 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a423292-136a-403f-8df3-fcef9bc1e7b9" path="/var/lib/kubelet/pods/5a423292-136a-403f-8df3-fcef9bc1e7b9/volumes" Jul 16 12:29:30.026754 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.026721 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q"] Jul 16 12:29:30.027228 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.027021 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a423292-136a-403f-8df3-fcef9bc1e7b9" containerName="manager" Jul 16 12:29:30.027228 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.027031 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a423292-136a-403f-8df3-fcef9bc1e7b9" containerName="manager" Jul 16 12:29:30.027228 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.027078 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a423292-136a-403f-8df3-fcef9bc1e7b9" containerName="manager" Jul 16 12:29:30.029957 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.029930 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" Jul 16 12:29:30.031987 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.031963 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 16 12:29:30.032146 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.032125 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nttp2\"" Jul 16 12:29:30.032215 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.032143 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 16 12:29:30.032981 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.032963 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 12:29:30.042069 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.042035 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q"] Jul 16 12:29:30.066184 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.066160 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w85z\" (UniqueName: \"kubernetes.io/projected/929d5fab-6f4e-4c32-b721-c5d207b77982-kube-api-access-4w85z\") pod \"maas-api-584d9dd8c9-pnc6q\" (UID: \"929d5fab-6f4e-4c32-b721-c5d207b77982\") " pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" Jul 16 12:29:30.066387 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.066195 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/929d5fab-6f4e-4c32-b721-c5d207b77982-openshift-service-ca\") pod \"maas-api-584d9dd8c9-pnc6q\" (UID: \"929d5fab-6f4e-4c32-b721-c5d207b77982\") " pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" Jul 16 12:29:30.066387 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.066221 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/929d5fab-6f4e-4c32-b721-c5d207b77982-maas-api-tls\") pod \"maas-api-584d9dd8c9-pnc6q\" (UID: \"929d5fab-6f4e-4c32-b721-c5d207b77982\") " pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" Jul 16 12:29:30.167175 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.167144 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w85z\" (UniqueName: \"kubernetes.io/projected/929d5fab-6f4e-4c32-b721-c5d207b77982-kube-api-access-4w85z\") pod \"maas-api-584d9dd8c9-pnc6q\" (UID: \"929d5fab-6f4e-4c32-b721-c5d207b77982\") " pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" Jul 16 12:29:30.167371 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.167184 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/929d5fab-6f4e-4c32-b721-c5d207b77982-openshift-service-ca\") pod \"maas-api-584d9dd8c9-pnc6q\" (UID: \"929d5fab-6f4e-4c32-b721-c5d207b77982\") " pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" Jul 16 12:29:30.167371 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.167212 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/929d5fab-6f4e-4c32-b721-c5d207b77982-maas-api-tls\") pod \"maas-api-584d9dd8c9-pnc6q\" (UID: \"929d5fab-6f4e-4c32-b721-c5d207b77982\") " pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" Jul 16 12:29:30.167801 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.167780 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/929d5fab-6f4e-4c32-b721-c5d207b77982-openshift-service-ca\") pod \"maas-api-584d9dd8c9-pnc6q\" (UID: \"929d5fab-6f4e-4c32-b721-c5d207b77982\") " pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" Jul 16 12:29:30.169824 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.169802 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/929d5fab-6f4e-4c32-b721-c5d207b77982-maas-api-tls\") pod \"maas-api-584d9dd8c9-pnc6q\" (UID: \"929d5fab-6f4e-4c32-b721-c5d207b77982\") " pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" Jul 16 12:29:30.177499 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.177473 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w85z\" (UniqueName: \"kubernetes.io/projected/929d5fab-6f4e-4c32-b721-c5d207b77982-kube-api-access-4w85z\") pod \"maas-api-584d9dd8c9-pnc6q\" (UID: \"929d5fab-6f4e-4c32-b721-c5d207b77982\") " pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" Jul 16 12:29:30.340172 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.340086 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" Jul 16 12:29:30.480608 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.480584 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q"] Jul 16 12:29:30.481616 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:29:30.481593 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod929d5fab_6f4e_4c32_b721_c5d207b77982.slice/crio-cd161d423e02d2c920c94c1cf9975048ce4e884e7430c00fb83cd11c51e90594 WatchSource:0}: Error finding container cd161d423e02d2c920c94c1cf9975048ce4e884e7430c00fb83cd11c51e90594: Status 404 returned error can't find the container with id cd161d423e02d2c920c94c1cf9975048ce4e884e7430c00fb83cd11c51e90594 Jul 16 12:29:30.546778 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.546748 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-6p265"] Jul 16 12:29:30.551068 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.551050 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-6p265" Jul 16 12:29:30.553355 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.553327 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 16 12:29:30.553487 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.553387 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dvntz\"" Jul 16 12:29:30.558863 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.558822 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-6p265"] Jul 16 12:29:30.672008 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.671982 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjbzk\" (UniqueName: \"kubernetes.io/projected/8218c232-5e0c-445d-aa1e-b7bff65ecb6d-kube-api-access-wjbzk\") pod \"payload-processing-f9b8d9b9b-6p265\" (UID: \"8218c232-5e0c-445d-aa1e-b7bff65ecb6d\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6p265" Jul 16 12:29:30.672176 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.672023 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8218c232-5e0c-445d-aa1e-b7bff65ecb6d-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-6p265\" (UID: \"8218c232-5e0c-445d-aa1e-b7bff65ecb6d\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6p265" Jul 16 12:29:30.773377 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.773319 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8218c232-5e0c-445d-aa1e-b7bff65ecb6d-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-6p265\" (UID: \"8218c232-5e0c-445d-aa1e-b7bff65ecb6d\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6p265" Jul 16 12:29:30.773533 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.773411 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjbzk\" (UniqueName: \"kubernetes.io/projected/8218c232-5e0c-445d-aa1e-b7bff65ecb6d-kube-api-access-wjbzk\") pod \"payload-processing-f9b8d9b9b-6p265\" (UID: \"8218c232-5e0c-445d-aa1e-b7bff65ecb6d\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6p265" Jul 16 12:29:30.773999 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.773978 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8218c232-5e0c-445d-aa1e-b7bff65ecb6d-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-6p265\" (UID: \"8218c232-5e0c-445d-aa1e-b7bff65ecb6d\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6p265" Jul 16 12:29:30.781987 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.781960 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjbzk\" (UniqueName: \"kubernetes.io/projected/8218c232-5e0c-445d-aa1e-b7bff65ecb6d-kube-api-access-wjbzk\") pod \"payload-processing-f9b8d9b9b-6p265\" (UID: \"8218c232-5e0c-445d-aa1e-b7bff65ecb6d\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-6p265" Jul 16 12:29:30.862033 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.862007 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-6p265" Jul 16 12:29:30.969764 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.969718 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" event={"ID":"929d5fab-6f4e-4c32-b721-c5d207b77982","Type":"ContainerStarted","Data":"cd161d423e02d2c920c94c1cf9975048ce4e884e7430c00fb83cd11c51e90594"} Jul 16 12:29:30.995047 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:30.995017 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-6p265"] Jul 16 12:29:30.997012 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:29:30.996986 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8218c232_5e0c_445d_aa1e_b7bff65ecb6d.slice/crio-3f0976196e0b4e66865266904b89d5e342f67c5700abf90905a3b884f6275302 WatchSource:0}: Error finding container 3f0976196e0b4e66865266904b89d5e342f67c5700abf90905a3b884f6275302: Status 404 returned error can't find the container with id 3f0976196e0b4e66865266904b89d5e342f67c5700abf90905a3b884f6275302 Jul 16 12:29:31.980132 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:31.980089 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-6p265" event={"ID":"8218c232-5e0c-445d-aa1e-b7bff65ecb6d","Type":"ContainerStarted","Data":"3f0976196e0b4e66865266904b89d5e342f67c5700abf90905a3b884f6275302"} Jul 16 12:29:32.989515 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:32.989478 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" event={"ID":"929d5fab-6f4e-4c32-b721-c5d207b77982","Type":"ContainerStarted","Data":"40259cd70fdf5947db3a1cb62e620a783b964ad50ada2b8bf991385cffc886fa"} Jul 16 12:29:32.989968 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:32.989673 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" Jul 16 12:29:32.991505 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:32.991478 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-6p265" event={"ID":"8218c232-5e0c-445d-aa1e-b7bff65ecb6d","Type":"ContainerStarted","Data":"b35659390afd3e0cea0f1732c468c19e6c92f798605a0ed795f6ee7fa2790e33"} Jul 16 12:29:32.991621 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:32.991586 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-6p265" Jul 16 12:29:33.009707 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:33.009668 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" podStartSLOduration=0.697651908 podStartE2EDuration="3.009649584s" totalImagesPullingTime="2.311997676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:29:33.007945187 +0000 UTC m=+536.869846433" watchObservedRunningTime="2026-07-16 12:29:33.009649584 +0000 UTC m=+536.871550829" Jul 16 12:29:33.024422 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:33.024379 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-6p265" podStartSLOduration=1.225708316 podStartE2EDuration="3.024366734s" totalImagesPullingTime="1.798658418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:29:33.022165373 +0000 UTC m=+536.884066637" watchObservedRunningTime="2026-07-16 12:29:33.024366734 +0000 UTC m=+536.886267978" Jul 16 12:29:38.019093 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:38.019063 2577 generic.go:356] "Generic (PLEG): container finished" podID="46882479-a6f2-48fe-8886-a81bbf1d101e" containerID="fa20ec0e09548d8c7f4783183c3ba5292edf56c5ec2702100023ffe51099ad65" exitCode=0 Jul 16 12:29:38.019472 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:38.019135 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" event={"ID":"46882479-a6f2-48fe-8886-a81bbf1d101e","Type":"ContainerDied","Data":"fa20ec0e09548d8c7f4783183c3ba5292edf56c5ec2702100023ffe51099ad65"} Jul 16 12:29:38.019578 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:38.019561 2577 scope.go:117] "RemoveContainer" containerID="fa20ec0e09548d8c7f4783183c3ba5292edf56c5ec2702100023ffe51099ad65" Jul 16 12:29:38.020579 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:38.020553 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:29:38.020637 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:38.020620 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:29:38.020672 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:38.020651 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:29:39.002789 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:39.002763 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-584d9dd8c9-pnc6q" Jul 16 12:29:39.025897 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:39.025869 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" event={"ID":"46882479-a6f2-48fe-8886-a81bbf1d101e","Type":"ContainerStarted","Data":"369b437e7553426c978c25dde87bc405a0bfb3f3ad335b6bf540ca4a7019a9a4"} Jul 16 12:29:39.845475 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:39.845440 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:29:39.846827 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:39.846792 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fh8dn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 16 12:29:39.846955 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:39.846850 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" podUID="46882479-a6f2-48fe-8886-a81bbf1d101e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 16 12:29:40.584647 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:40.584590 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:29:40.845286 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:40.845212 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fh8dn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 16 12:29:40.845471 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:40.845270 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" podUID="46882479-a6f2-48fe-8886-a81bbf1d101e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 16 12:29:41.845179 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:41.845138 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fh8dn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 16 12:29:41.845678 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:41.845211 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" podUID="46882479-a6f2-48fe-8886-a81bbf1d101e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 16 12:29:42.845370 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:42.845331 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-fh8dn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 16 12:29:42.845736 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:42.845385 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" podUID="46882479-a6f2-48fe-8886-a81bbf1d101e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 16 12:29:43.848383 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:43.848354 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:29:43.849369 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:43.849349 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-fh8dn" Jul 16 12:29:43.999077 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:29:43.999049 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-6p265" Jul 16 12:30:22.918311 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:22.918260 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s"] Jul 16 12:30:22.923565 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:22.923546 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:22.926516 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:22.926483 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-78wqk\"" Jul 16 12:30:22.926516 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:22.926515 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 12:30:22.926711 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:22.926518 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 16 12:30:22.926711 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:22.926556 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 12:30:22.930353 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:22.930331 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s"] Jul 16 12:30:23.030872 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.030832 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.031046 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.030894 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg7ml\" (UniqueName: \"kubernetes.io/projected/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-kube-api-access-qg7ml\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.031046 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.030956 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.031046 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.031006 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.031046 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.031024 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.031214 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.031051 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.132020 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.131987 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.132189 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.132044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.132189 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.132106 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg7ml\" (UniqueName: \"kubernetes.io/projected/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-kube-api-access-qg7ml\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.132189 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.132138 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.132189 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.132176 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.132437 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.132199 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.132627 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.132595 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.132627 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.132601 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.132810 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.132639 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.134397 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.134370 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.134720 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.134703 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.140100 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.140078 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg7ml\" (UniqueName: \"kubernetes.io/projected/6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc-kube-api-access-qg7ml\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s\" (UID: \"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.235191 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.235105 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:23.365805 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:23.365782 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s"] Jul 16 12:30:23.367850 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:30:23.367823 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e2f730c_9a5f_47c3_bc71_8f0ca4c6bedc.slice/crio-ec53716cc8fe6fe1d8597a2688aad447e6af4389439d61309641338e1ee1f7ea WatchSource:0}: Error finding container ec53716cc8fe6fe1d8597a2688aad447e6af4389439d61309641338e1ee1f7ea: Status 404 returned error can't find the container with id ec53716cc8fe6fe1d8597a2688aad447e6af4389439d61309641338e1ee1f7ea Jul 16 12:30:24.252692 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:24.252655 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" event={"ID":"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc","Type":"ContainerStarted","Data":"ec53716cc8fe6fe1d8597a2688aad447e6af4389439d61309641338e1ee1f7ea"} Jul 16 12:30:25.269637 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:25.269444 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" event={"ID":"6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc","Type":"ContainerStarted","Data":"14d35a26fe47f4d4ce89fa4f760dead6edb25c2447347edad0bd7e4fffdd03f3"} Jul 16 12:30:25.287572 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:25.287522 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" podStartSLOduration=2.242894246 podStartE2EDuration="3.287506408s" totalImagesPullingTime="1.044612162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:30:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:30:25.286880434 +0000 UTC m=+589.148781680" watchObservedRunningTime="2026-07-16 12:30:25.287506408 +0000 UTC m=+589.149407653" Jul 16 12:30:33.236223 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:33.236179 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:33.236735 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:33.236258 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:33.250545 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:33.250516 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:33.321213 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:33.321190 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s" Jul 16 12:30:36.520167 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:36.520139 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:30:36.521267 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:30:36.521244 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:33:02.868648 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:02.868612 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74b78db6b9-zqz6s"] Jul 16 12:33:02.869103 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:02.868833 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74b78db6b9-zqz6s" podUID="083698c4-8478-4fb5-9653-365d4d8a5a4e" containerName="manager" containerID="cri-o://92c88a2e25bb17d461c8e6de86a5e30a9358648b0491d258bd625698f893a4d4" gracePeriod=30 Jul 16 12:33:03.055070 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:03.055042 2577 generic.go:356] "Generic (PLEG): container finished" podID="083698c4-8478-4fb5-9653-365d4d8a5a4e" containerID="92c88a2e25bb17d461c8e6de86a5e30a9358648b0491d258bd625698f893a4d4" exitCode=0 Jul 16 12:33:03.055204 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:03.055086 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b78db6b9-zqz6s" event={"ID":"083698c4-8478-4fb5-9653-365d4d8a5a4e","Type":"ContainerDied","Data":"92c88a2e25bb17d461c8e6de86a5e30a9358648b0491d258bd625698f893a4d4"} Jul 16 12:33:03.120587 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:03.120525 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b78db6b9-zqz6s" Jul 16 12:33:03.248775 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:03.248693 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/083698c4-8478-4fb5-9653-365d4d8a5a4e-webhook-cert\") pod \"083698c4-8478-4fb5-9653-365d4d8a5a4e\" (UID: \"083698c4-8478-4fb5-9653-365d4d8a5a4e\") " Jul 16 12:33:03.248972 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:03.248794 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzdrr\" (UniqueName: \"kubernetes.io/projected/083698c4-8478-4fb5-9653-365d4d8a5a4e-kube-api-access-tzdrr\") pod \"083698c4-8478-4fb5-9653-365d4d8a5a4e\" (UID: \"083698c4-8478-4fb5-9653-365d4d8a5a4e\") " Jul 16 12:33:03.250909 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:03.250879 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/083698c4-8478-4fb5-9653-365d4d8a5a4e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "083698c4-8478-4fb5-9653-365d4d8a5a4e" (UID: "083698c4-8478-4fb5-9653-365d4d8a5a4e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:33:03.251042 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:03.251020 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/083698c4-8478-4fb5-9653-365d4d8a5a4e-kube-api-access-tzdrr" (OuterVolumeSpecName: "kube-api-access-tzdrr") pod "083698c4-8478-4fb5-9653-365d4d8a5a4e" (UID: "083698c4-8478-4fb5-9653-365d4d8a5a4e"). InnerVolumeSpecName "kube-api-access-tzdrr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:33:03.349713 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:03.349675 2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/083698c4-8478-4fb5-9653-365d4d8a5a4e-webhook-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:33:03.349713 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:03.349704 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzdrr\" (UniqueName: \"kubernetes.io/projected/083698c4-8478-4fb5-9653-365d4d8a5a4e-kube-api-access-tzdrr\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:33:04.060950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:04.060914 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b78db6b9-zqz6s" event={"ID":"083698c4-8478-4fb5-9653-365d4d8a5a4e","Type":"ContainerDied","Data":"58f39f59803b345acdb0d051898eb24a2b3bc767986bb091a3c7b615a63349fd"} Jul 16 12:33:04.060950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:04.060944 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b78db6b9-zqz6s" Jul 16 12:33:04.061532 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:04.060974 2577 scope.go:117] "RemoveContainer" containerID="92c88a2e25bb17d461c8e6de86a5e30a9358648b0491d258bd625698f893a4d4" Jul 16 12:33:04.083047 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:04.083013 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74b78db6b9-zqz6s"] Jul 16 12:33:04.088576 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:04.088545 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74b78db6b9-zqz6s"] Jul 16 12:33:04.638199 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:33:04.638162 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="083698c4-8478-4fb5-9653-365d4d8a5a4e" path="/var/lib/kubelet/pods/083698c4-8478-4fb5-9653-365d4d8a5a4e/volumes" Jul 16 12:34:04.984541 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:04.984465 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74b78db6b9-t2pl2"] Jul 16 12:34:04.984940 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:04.984777 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="083698c4-8478-4fb5-9653-365d4d8a5a4e" containerName="manager" Jul 16 12:34:04.984940 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:04.984788 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="083698c4-8478-4fb5-9653-365d4d8a5a4e" containerName="manager" Jul 16 12:34:04.984940 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:04.984845 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="083698c4-8478-4fb5-9653-365d4d8a5a4e" containerName="manager" Jul 16 12:34:04.987708 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:04.987691 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b78db6b9-t2pl2" Jul 16 12:34:04.991400 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:04.991368 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bhcp6\"" Jul 16 12:34:04.991400 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:04.991379 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 12:34:04.991551 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:04.991381 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 12:34:05.004574 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:05.004553 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74b78db6b9-t2pl2"] Jul 16 12:34:05.088280 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:05.088240 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-552h9\" (UniqueName: \"kubernetes.io/projected/b7671641-9b93-49ed-a891-2cbea5edf55d-kube-api-access-552h9\") pod \"maas-controller-74b78db6b9-t2pl2\" (UID: \"b7671641-9b93-49ed-a891-2cbea5edf55d\") " pod="opendatahub/maas-controller-74b78db6b9-t2pl2" Jul 16 12:34:05.088280 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:05.088284 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7671641-9b93-49ed-a891-2cbea5edf55d-webhook-cert\") pod \"maas-controller-74b78db6b9-t2pl2\" (UID: \"b7671641-9b93-49ed-a891-2cbea5edf55d\") " pod="opendatahub/maas-controller-74b78db6b9-t2pl2" Jul 16 12:34:05.188929 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:05.188882 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-552h9\" (UniqueName: \"kubernetes.io/projected/b7671641-9b93-49ed-a891-2cbea5edf55d-kube-api-access-552h9\") pod \"maas-controller-74b78db6b9-t2pl2\" (UID: \"b7671641-9b93-49ed-a891-2cbea5edf55d\") " pod="opendatahub/maas-controller-74b78db6b9-t2pl2" Jul 16 12:34:05.188929 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:05.188934 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7671641-9b93-49ed-a891-2cbea5edf55d-webhook-cert\") pod \"maas-controller-74b78db6b9-t2pl2\" (UID: \"b7671641-9b93-49ed-a891-2cbea5edf55d\") " pod="opendatahub/maas-controller-74b78db6b9-t2pl2" Jul 16 12:34:05.191473 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:05.191450 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7671641-9b93-49ed-a891-2cbea5edf55d-webhook-cert\") pod \"maas-controller-74b78db6b9-t2pl2\" (UID: \"b7671641-9b93-49ed-a891-2cbea5edf55d\") " pod="opendatahub/maas-controller-74b78db6b9-t2pl2" Jul 16 12:34:05.205100 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:05.205066 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-552h9\" (UniqueName: \"kubernetes.io/projected/b7671641-9b93-49ed-a891-2cbea5edf55d-kube-api-access-552h9\") pod \"maas-controller-74b78db6b9-t2pl2\" (UID: \"b7671641-9b93-49ed-a891-2cbea5edf55d\") " pod="opendatahub/maas-controller-74b78db6b9-t2pl2" Jul 16 12:34:05.299119 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:05.299024 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b78db6b9-t2pl2" Jul 16 12:34:05.442139 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:05.442110 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74b78db6b9-t2pl2"] Jul 16 12:34:05.444091 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:34:05.444059 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7671641_9b93_49ed_a891_2cbea5edf55d.slice/crio-c7109b5a1b01cbbd2408119faaf5d5e4ddcfc4b81a7f4a98df32f8b47569120a WatchSource:0}: Error finding container c7109b5a1b01cbbd2408119faaf5d5e4ddcfc4b81a7f4a98df32f8b47569120a: Status 404 returned error can't find the container with id c7109b5a1b01cbbd2408119faaf5d5e4ddcfc4b81a7f4a98df32f8b47569120a Jul 16 12:34:05.445501 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:05.445482 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 12:34:06.377250 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:06.377161 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b78db6b9-t2pl2" event={"ID":"b7671641-9b93-49ed-a891-2cbea5edf55d","Type":"ContainerStarted","Data":"5969c410d11e5b7d7e4d3598addbabfdf6fdef6b979fd2ada013ff0599b3d3ca"} Jul 16 12:34:06.377250 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:06.377216 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74b78db6b9-t2pl2" Jul 16 12:34:06.377250 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:06.377233 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b78db6b9-t2pl2" event={"ID":"b7671641-9b93-49ed-a891-2cbea5edf55d","Type":"ContainerStarted","Data":"c7109b5a1b01cbbd2408119faaf5d5e4ddcfc4b81a7f4a98df32f8b47569120a"} Jul 16 12:34:06.397528 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:06.397476 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74b78db6b9-t2pl2" podStartSLOduration=1.7781986330000001 podStartE2EDuration="2.397460595s" totalImagesPullingTime="619.261962ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:34:06.396987118 +0000 UTC m=+810.258888378" watchObservedRunningTime="2026-07-16 12:34:06.397460595 +0000 UTC m=+810.259361840" Jul 16 12:34:17.385160 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:17.385126 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74b78db6b9-t2pl2" Jul 16 12:34:22.023104 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.023065 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74b78db6b9-t2pl2"] Jul 16 12:34:22.023599 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.023277 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74b78db6b9-t2pl2" podUID="b7671641-9b93-49ed-a891-2cbea5edf55d" containerName="manager" containerID="cri-o://5969c410d11e5b7d7e4d3598addbabfdf6fdef6b979fd2ada013ff0599b3d3ca" gracePeriod=30 Jul 16 12:34:22.273802 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.273746 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b78db6b9-t2pl2" Jul 16 12:34:22.327949 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.327921 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7671641-9b93-49ed-a891-2cbea5edf55d-webhook-cert\") pod \"b7671641-9b93-49ed-a891-2cbea5edf55d\" (UID: \"b7671641-9b93-49ed-a891-2cbea5edf55d\") " Jul 16 12:34:22.328108 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.327991 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-552h9\" (UniqueName: \"kubernetes.io/projected/b7671641-9b93-49ed-a891-2cbea5edf55d-kube-api-access-552h9\") pod \"b7671641-9b93-49ed-a891-2cbea5edf55d\" (UID: \"b7671641-9b93-49ed-a891-2cbea5edf55d\") " Jul 16 12:34:22.330008 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.329972 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7671641-9b93-49ed-a891-2cbea5edf55d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b7671641-9b93-49ed-a891-2cbea5edf55d" (UID: "b7671641-9b93-49ed-a891-2cbea5edf55d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 12:34:22.330190 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.330173 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7671641-9b93-49ed-a891-2cbea5edf55d-kube-api-access-552h9" (OuterVolumeSpecName: "kube-api-access-552h9") pod "b7671641-9b93-49ed-a891-2cbea5edf55d" (UID: "b7671641-9b93-49ed-a891-2cbea5edf55d"). InnerVolumeSpecName "kube-api-access-552h9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:34:22.428593 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.428560 2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b7671641-9b93-49ed-a891-2cbea5edf55d-webhook-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:34:22.428593 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.428588 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-552h9\" (UniqueName: \"kubernetes.io/projected/b7671641-9b93-49ed-a891-2cbea5edf55d-kube-api-access-552h9\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:34:22.458174 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.458136 2577 generic.go:356] "Generic (PLEG): container finished" podID="b7671641-9b93-49ed-a891-2cbea5edf55d" containerID="5969c410d11e5b7d7e4d3598addbabfdf6fdef6b979fd2ada013ff0599b3d3ca" exitCode=0 Jul 16 12:34:22.458354 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.458185 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b78db6b9-t2pl2" Jul 16 12:34:22.458354 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.458231 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b78db6b9-t2pl2" event={"ID":"b7671641-9b93-49ed-a891-2cbea5edf55d","Type":"ContainerDied","Data":"5969c410d11e5b7d7e4d3598addbabfdf6fdef6b979fd2ada013ff0599b3d3ca"} Jul 16 12:34:22.458354 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.458313 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b78db6b9-t2pl2" event={"ID":"b7671641-9b93-49ed-a891-2cbea5edf55d","Type":"ContainerDied","Data":"c7109b5a1b01cbbd2408119faaf5d5e4ddcfc4b81a7f4a98df32f8b47569120a"} Jul 16 12:34:22.458354 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.458344 2577 scope.go:117] "RemoveContainer" containerID="5969c410d11e5b7d7e4d3598addbabfdf6fdef6b979fd2ada013ff0599b3d3ca" Jul 16 12:34:22.467219 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.467203 2577 scope.go:117] "RemoveContainer" containerID="5969c410d11e5b7d7e4d3598addbabfdf6fdef6b979fd2ada013ff0599b3d3ca" Jul 16 12:34:22.467468 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:34:22.467452 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5969c410d11e5b7d7e4d3598addbabfdf6fdef6b979fd2ada013ff0599b3d3ca\": container with ID starting with 5969c410d11e5b7d7e4d3598addbabfdf6fdef6b979fd2ada013ff0599b3d3ca not found: ID does not exist" containerID="5969c410d11e5b7d7e4d3598addbabfdf6fdef6b979fd2ada013ff0599b3d3ca" Jul 16 12:34:22.467527 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.467478 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5969c410d11e5b7d7e4d3598addbabfdf6fdef6b979fd2ada013ff0599b3d3ca"} err="failed to get container status \"5969c410d11e5b7d7e4d3598addbabfdf6fdef6b979fd2ada013ff0599b3d3ca\": rpc error: code = NotFound desc = could not find container \"5969c410d11e5b7d7e4d3598addbabfdf6fdef6b979fd2ada013ff0599b3d3ca\": container with ID starting with 5969c410d11e5b7d7e4d3598addbabfdf6fdef6b979fd2ada013ff0599b3d3ca not found: ID does not exist" Jul 16 12:34:22.480130 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.480105 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74b78db6b9-t2pl2"] Jul 16 12:34:22.484371 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.484351 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74b78db6b9-t2pl2"] Jul 16 12:34:22.638703 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:22.638632 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7671641-9b93-49ed-a891-2cbea5edf55d" path="/var/lib/kubelet/pods/b7671641-9b93-49ed-a891-2cbea5edf55d/volumes" Jul 16 12:34:23.269525 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.269493 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74b78db6b9-ljb2m"] Jul 16 12:34:23.269896 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.269841 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7671641-9b93-49ed-a891-2cbea5edf55d" containerName="manager" Jul 16 12:34:23.269896 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.269853 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7671641-9b93-49ed-a891-2cbea5edf55d" containerName="manager" Jul 16 12:34:23.269966 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.269912 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7671641-9b93-49ed-a891-2cbea5edf55d" containerName="manager" Jul 16 12:34:23.274063 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.274049 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b78db6b9-ljb2m" Jul 16 12:34:23.277157 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.277137 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bhcp6\"" Jul 16 12:34:23.277327 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.277139 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 12:34:23.277327 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.277139 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 12:34:23.281976 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.281954 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74b78db6b9-ljb2m"] Jul 16 12:34:23.335828 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.335797 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4r8b\" (UniqueName: \"kubernetes.io/projected/76469e9c-8006-46b4-9a56-e4a02be42384-kube-api-access-v4r8b\") pod \"maas-controller-74b78db6b9-ljb2m\" (UID: \"76469e9c-8006-46b4-9a56-e4a02be42384\") " pod="opendatahub/maas-controller-74b78db6b9-ljb2m" Jul 16 12:34:23.336019 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.335850 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/76469e9c-8006-46b4-9a56-e4a02be42384-webhook-cert\") pod \"maas-controller-74b78db6b9-ljb2m\" (UID: \"76469e9c-8006-46b4-9a56-e4a02be42384\") " pod="opendatahub/maas-controller-74b78db6b9-ljb2m" Jul 16 12:34:23.436673 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.436636 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4r8b\" (UniqueName: \"kubernetes.io/projected/76469e9c-8006-46b4-9a56-e4a02be42384-kube-api-access-v4r8b\") pod \"maas-controller-74b78db6b9-ljb2m\" (UID: \"76469e9c-8006-46b4-9a56-e4a02be42384\") " pod="opendatahub/maas-controller-74b78db6b9-ljb2m" Jul 16 12:34:23.436860 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.436686 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/76469e9c-8006-46b4-9a56-e4a02be42384-webhook-cert\") pod \"maas-controller-74b78db6b9-ljb2m\" (UID: \"76469e9c-8006-46b4-9a56-e4a02be42384\") " pod="opendatahub/maas-controller-74b78db6b9-ljb2m" Jul 16 12:34:23.439285 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.439262 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/76469e9c-8006-46b4-9a56-e4a02be42384-webhook-cert\") pod \"maas-controller-74b78db6b9-ljb2m\" (UID: \"76469e9c-8006-46b4-9a56-e4a02be42384\") " pod="opendatahub/maas-controller-74b78db6b9-ljb2m" Jul 16 12:34:23.445778 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.445753 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4r8b\" (UniqueName: \"kubernetes.io/projected/76469e9c-8006-46b4-9a56-e4a02be42384-kube-api-access-v4r8b\") pod \"maas-controller-74b78db6b9-ljb2m\" (UID: \"76469e9c-8006-46b4-9a56-e4a02be42384\") " pod="opendatahub/maas-controller-74b78db6b9-ljb2m" Jul 16 12:34:23.585184 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.585094 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74b78db6b9-ljb2m" Jul 16 12:34:23.717246 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:23.717127 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74b78db6b9-ljb2m"] Jul 16 12:34:23.719760 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:34:23.719736 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76469e9c_8006_46b4_9a56_e4a02be42384.slice/crio-316fbdddae30e252a38eff4b2304925717347ee03b909013d8461adb2393d6f7 WatchSource:0}: Error finding container 316fbdddae30e252a38eff4b2304925717347ee03b909013d8461adb2393d6f7: Status 404 returned error can't find the container with id 316fbdddae30e252a38eff4b2304925717347ee03b909013d8461adb2393d6f7 Jul 16 12:34:24.470941 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:24.470889 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b78db6b9-ljb2m" event={"ID":"76469e9c-8006-46b4-9a56-e4a02be42384","Type":"ContainerStarted","Data":"993d7e7559f68d1140450dcfc82b5739fc2e873ad1b602f135ffe2b82156eb1c"} Jul 16 12:34:24.471354 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:24.470995 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74b78db6b9-ljb2m" event={"ID":"76469e9c-8006-46b4-9a56-e4a02be42384","Type":"ContainerStarted","Data":"316fbdddae30e252a38eff4b2304925717347ee03b909013d8461adb2393d6f7"} Jul 16 12:34:24.471354 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:24.471029 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74b78db6b9-ljb2m" Jul 16 12:34:24.491133 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:24.491082 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74b78db6b9-ljb2m" podStartSLOduration=0.874342801 podStartE2EDuration="1.491068198s" totalImagesPullingTime="616.725397ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:34:24.490248673 +0000 UTC m=+828.352149921" watchObservedRunningTime="2026-07-16 12:34:24.491068198 +0000 UTC m=+828.352969443" Jul 16 12:34:35.478127 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:34:35.478096 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74b78db6b9-ljb2m" Jul 16 12:35:36.550120 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:35:36.550093 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:35:36.551515 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:35:36.551490 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:40:36.579812 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:40:36.579781 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:40:36.582652 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:40:36.582630 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:45:00.138032 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:00.137996 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736765-7rxbc"] Jul 16 12:45:00.141508 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:00.141487 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736765-7rxbc" Jul 16 12:45:00.143565 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:00.143548 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zjpqc\"" Jul 16 12:45:00.150507 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:00.150489 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736765-7rxbc" Jul 16 12:45:00.161122 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:00.161100 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736765-7rxbc"] Jul 16 12:45:00.286933 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:00.286906 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736765-7rxbc"] Jul 16 12:45:00.288487 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:45:00.288459 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3caad597_6c8a_40d5_97e3_4dde3f8b162c.slice/crio-ae48df9c98bacfed1b31aef2254141a25403fd66b86cc42372cdccf08701fdf5 WatchSource:0}: Error finding container ae48df9c98bacfed1b31aef2254141a25403fd66b86cc42372cdccf08701fdf5: Status 404 returned error can't find the container with id ae48df9c98bacfed1b31aef2254141a25403fd66b86cc42372cdccf08701fdf5 Jul 16 12:45:00.290765 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:00.290744 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 12:45:00.608037 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:00.608005 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736765-7rxbc" event={"ID":"3caad597-6c8a-40d5-97e3-4dde3f8b162c","Type":"ContainerStarted","Data":"ae48df9c98bacfed1b31aef2254141a25403fd66b86cc42372cdccf08701fdf5"} Jul 16 12:45:03.625866 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:03.625834 2577 generic.go:356] "Generic (PLEG): container finished" podID="3caad597-6c8a-40d5-97e3-4dde3f8b162c" containerID="74b9bd10c832507840b37af0a781be6738efb93a8267f92aac03c738d25201d1" exitCode=0 Jul 16 12:45:03.626239 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:03.625905 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736765-7rxbc" event={"ID":"3caad597-6c8a-40d5-97e3-4dde3f8b162c","Type":"ContainerDied","Data":"74b9bd10c832507840b37af0a781be6738efb93a8267f92aac03c738d25201d1"} Jul 16 12:45:04.766928 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:04.766902 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736765-7rxbc" Jul 16 12:45:05.637216 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:05.637183 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736765-7rxbc" event={"ID":"3caad597-6c8a-40d5-97e3-4dde3f8b162c","Type":"ContainerDied","Data":"ae48df9c98bacfed1b31aef2254141a25403fd66b86cc42372cdccf08701fdf5"} Jul 16 12:45:05.637479 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:05.637220 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736765-7rxbc" Jul 16 12:45:05.637479 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:05.637232 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae48df9c98bacfed1b31aef2254141a25403fd66b86cc42372cdccf08701fdf5" Jul 16 12:45:36.614618 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:36.614502 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:45:36.626936 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:45:36.626908 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:46:13.245459 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.245376 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv"] Jul 16 12:46:13.246008 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.245694 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3caad597-6c8a-40d5-97e3-4dde3f8b162c" containerName="cleanup" Jul 16 12:46:13.246008 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.245704 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3caad597-6c8a-40d5-97e3-4dde3f8b162c" containerName="cleanup" Jul 16 12:46:13.246008 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.245774 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3caad597-6c8a-40d5-97e3-4dde3f8b162c" containerName="cleanup" Jul 16 12:46:13.248887 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.248862 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.251084 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.251065 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-767373bc-openshift-default-dockercfg-hvr7f\"" Jul 16 12:46:13.264435 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.264410 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv"] Jul 16 12:46:13.348174 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.348140 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-podinfo\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.348174 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.348174 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-token\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.348448 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.348210 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-credential-socket\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.348448 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.348226 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-workload-certs\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.348448 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.348253 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5qfk\" (UniqueName: \"kubernetes.io/projected/89970cf0-95c2-42e9-9b84-6e8b22854227-kube-api-access-z5qfk\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.348448 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.348351 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-envoy\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.348448 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.348402 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-workload-socket\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.348706 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.348464 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-data\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.348706 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.348491 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89970cf0-95c2-42e9-9b84-6e8b22854227-istiod-ca-cert\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.449990 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.449950 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-envoy\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.450169 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.450012 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-workload-socket\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.450169 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.450067 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-data\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.450169 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.450090 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89970cf0-95c2-42e9-9b84-6e8b22854227-istiod-ca-cert\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.450169 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.450146 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-podinfo\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.450409 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.450168 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-token\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.450409 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.450235 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-credential-socket\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.450409 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.450260 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-workload-certs\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.450409 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.450339 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5qfk\" (UniqueName: \"kubernetes.io/projected/89970cf0-95c2-42e9-9b84-6e8b22854227-kube-api-access-z5qfk\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.450617 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.450536 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-workload-socket\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.450688 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.450627 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-credential-socket\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.450795 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.450769 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-workload-certs\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.451143 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.451122 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-data\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.451412 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.451391 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89970cf0-95c2-42e9-9b84-6e8b22854227-istiod-ca-cert\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.452504 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.452484 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-envoy\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.452985 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.452967 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-podinfo\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.462126 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.462103 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-token\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.462339 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.462316 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5qfk\" (UniqueName: \"kubernetes.io/projected/89970cf0-95c2-42e9-9b84-6e8b22854227-kube-api-access-z5qfk\") pod \"e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.524421 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.524343 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824"] Jul 16 12:46:13.528687 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.528671 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:13.531459 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.531440 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-767373bc-serving-cert\"" Jul 16 12:46:13.539149 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.539122 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824"] Jul 16 12:46:13.558658 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.558624 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:13.652823 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.652792 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjkvs\" (UniqueName: \"kubernetes.io/projected/aedee2fb-4228-418a-857b-33e714c2baea-kube-api-access-cjkvs\") pod \"maas-api-e2e-ait-767373bc-5ff4467c86-lk824\" (UID: \"aedee2fb-4228-418a-857b-33e714c2baea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:13.652980 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.652918 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aedee2fb-4228-418a-857b-33e714c2baea-maas-api-tls\") pod \"maas-api-e2e-ait-767373bc-5ff4467c86-lk824\" (UID: \"aedee2fb-4228-418a-857b-33e714c2baea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:13.652980 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.652953 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aedee2fb-4228-418a-857b-33e714c2baea-openshift-service-ca\") pod \"maas-api-e2e-ait-767373bc-5ff4467c86-lk824\" (UID: \"aedee2fb-4228-418a-857b-33e714c2baea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:13.702630 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.702596 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv"] Jul 16 12:46:13.704043 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:46:13.704015 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89970cf0_95c2_42e9_9b84_6e8b22854227.slice/crio-c6ea87cb3421263def0e142522f3f27bcc88f16d53220cee42dcebc273a2a122 WatchSource:0}: Error finding container c6ea87cb3421263def0e142522f3f27bcc88f16d53220cee42dcebc273a2a122: Status 404 returned error can't find the container with id c6ea87cb3421263def0e142522f3f27bcc88f16d53220cee42dcebc273a2a122 Jul 16 12:46:13.706356 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.706321 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:46:13.706430 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.706404 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:46:13.706465 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.706442 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:46:13.753706 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.753668 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aedee2fb-4228-418a-857b-33e714c2baea-maas-api-tls\") pod \"maas-api-e2e-ait-767373bc-5ff4467c86-lk824\" (UID: \"aedee2fb-4228-418a-857b-33e714c2baea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:13.753813 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.753710 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aedee2fb-4228-418a-857b-33e714c2baea-openshift-service-ca\") pod \"maas-api-e2e-ait-767373bc-5ff4467c86-lk824\" (UID: \"aedee2fb-4228-418a-857b-33e714c2baea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:13.753813 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.753785 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjkvs\" (UniqueName: \"kubernetes.io/projected/aedee2fb-4228-418a-857b-33e714c2baea-kube-api-access-cjkvs\") pod \"maas-api-e2e-ait-767373bc-5ff4467c86-lk824\" (UID: \"aedee2fb-4228-418a-857b-33e714c2baea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:13.753923 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:46:13.753826 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-serving-cert: secret "maas-api-e2e-ait-767373bc-serving-cert" not found Jul 16 12:46:13.753923 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:46:13.753901 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aedee2fb-4228-418a-857b-33e714c2baea-maas-api-tls podName:aedee2fb-4228-418a-857b-33e714c2baea nodeName:}" failed. No retries permitted until 2026-07-16 12:46:14.253879527 +0000 UTC m=+1538.115780751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aedee2fb-4228-418a-857b-33e714c2baea-maas-api-tls") pod "maas-api-e2e-ait-767373bc-5ff4467c86-lk824" (UID: "aedee2fb-4228-418a-857b-33e714c2baea") : secret "maas-api-e2e-ait-767373bc-serving-cert" not found Jul 16 12:46:13.754450 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.754428 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aedee2fb-4228-418a-857b-33e714c2baea-openshift-service-ca\") pod \"maas-api-e2e-ait-767373bc-5ff4467c86-lk824\" (UID: \"aedee2fb-4228-418a-857b-33e714c2baea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:13.762388 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.762366 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjkvs\" (UniqueName: \"kubernetes.io/projected/aedee2fb-4228-418a-857b-33e714c2baea-kube-api-access-cjkvs\") pod \"maas-api-e2e-ait-767373bc-5ff4467c86-lk824\" (UID: \"aedee2fb-4228-418a-857b-33e714c2baea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:13.966078 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.966035 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" event={"ID":"89970cf0-95c2-42e9-9b84-6e8b22854227","Type":"ContainerStarted","Data":"bac1e5a58ad503d7f1ae760f5a93d643535b06b2ed3a9058ac2f59a5027e99f5"} Jul 16 12:46:13.966274 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.966098 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" event={"ID":"89970cf0-95c2-42e9-9b84-6e8b22854227","Type":"ContainerStarted","Data":"c6ea87cb3421263def0e142522f3f27bcc88f16d53220cee42dcebc273a2a122"} Jul 16 12:46:13.997219 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:13.997169 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" podStartSLOduration=0.997151275 podStartE2EDuration="997.151275ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:46:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:46:13.995226369 +0000 UTC m=+1537.857127652" watchObservedRunningTime="2026-07-16 12:46:13.997151275 +0000 UTC m=+1537.859052519" Jul 16 12:46:14.258695 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:14.258596 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aedee2fb-4228-418a-857b-33e714c2baea-maas-api-tls\") pod \"maas-api-e2e-ait-767373bc-5ff4467c86-lk824\" (UID: \"aedee2fb-4228-418a-857b-33e714c2baea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:14.261335 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:14.261280 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aedee2fb-4228-418a-857b-33e714c2baea-maas-api-tls\") pod \"maas-api-e2e-ait-767373bc-5ff4467c86-lk824\" (UID: \"aedee2fb-4228-418a-857b-33e714c2baea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:14.439629 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:14.439594 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:14.559584 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:14.559554 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:14.564768 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:14.564742 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:14.592226 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:14.592159 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824"] Jul 16 12:46:14.596972 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:46:14.596946 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaedee2fb_4228_418a_857b_33e714c2baea.slice/crio-02318349c7bbbdb308cd65e90bce7ccf4bb1f9a2034931d9c11492339447b0e7 WatchSource:0}: Error finding container 02318349c7bbbdb308cd65e90bce7ccf4bb1f9a2034931d9c11492339447b0e7: Status 404 returned error can't find the container with id 02318349c7bbbdb308cd65e90bce7ccf4bb1f9a2034931d9c11492339447b0e7 Jul 16 12:46:14.972281 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:14.972237 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" event={"ID":"aedee2fb-4228-418a-857b-33e714c2baea","Type":"ContainerStarted","Data":"02318349c7bbbdb308cd65e90bce7ccf4bb1f9a2034931d9c11492339447b0e7"} Jul 16 12:46:14.972650 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:14.972627 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:14.973542 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:14.973523 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:15.978870 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:15.978832 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" event={"ID":"aedee2fb-4228-418a-857b-33e714c2baea","Type":"ContainerStarted","Data":"9ae93f7886c118aac2a62b309bb25f5af01321dab1da8c12f4b24c35ecc40c30"} Jul 16 12:46:15.979265 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:15.979080 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:16.000989 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:16.000936 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" podStartSLOduration=2.457042308 podStartE2EDuration="3.000921165s" totalImagesPullingTime="543.878857ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:46:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:46:15.998968047 +0000 UTC m=+1539.860869295" watchObservedRunningTime="2026-07-16 12:46:16.000921165 +0000 UTC m=+1539.862822410" Jul 16 12:46:19.723001 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:19.722971 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv"] Jul 16 12:46:19.723401 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:19.723142 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" podUID="89970cf0-95c2-42e9-9b84-6e8b22854227" containerName="istio-proxy" containerID="cri-o://bac1e5a58ad503d7f1ae760f5a93d643535b06b2ed3a9058ac2f59a5027e99f5" gracePeriod=30 Jul 16 12:46:21.990006 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:21.989976 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-767373bc-5ff4467c86-lk824" Jul 16 12:46:24.973439 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:24.973413 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:25.026240 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.026158 2577 generic.go:356] "Generic (PLEG): container finished" podID="89970cf0-95c2-42e9-9b84-6e8b22854227" containerID="bac1e5a58ad503d7f1ae760f5a93d643535b06b2ed3a9058ac2f59a5027e99f5" exitCode=0 Jul 16 12:46:25.026429 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.026256 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" Jul 16 12:46:25.026429 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.026320 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" event={"ID":"89970cf0-95c2-42e9-9b84-6e8b22854227","Type":"ContainerDied","Data":"bac1e5a58ad503d7f1ae760f5a93d643535b06b2ed3a9058ac2f59a5027e99f5"} Jul 16 12:46:25.026429 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.026383 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv" event={"ID":"89970cf0-95c2-42e9-9b84-6e8b22854227","Type":"ContainerDied","Data":"c6ea87cb3421263def0e142522f3f27bcc88f16d53220cee42dcebc273a2a122"} Jul 16 12:46:25.026429 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.026413 2577 scope.go:117] "RemoveContainer" containerID="bac1e5a58ad503d7f1ae760f5a93d643535b06b2ed3a9058ac2f59a5027e99f5" Jul 16 12:46:25.035171 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.035154 2577 scope.go:117] "RemoveContainer" containerID="bac1e5a58ad503d7f1ae760f5a93d643535b06b2ed3a9058ac2f59a5027e99f5" Jul 16 12:46:25.035470 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:46:25.035449 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bac1e5a58ad503d7f1ae760f5a93d643535b06b2ed3a9058ac2f59a5027e99f5\": container with ID starting with bac1e5a58ad503d7f1ae760f5a93d643535b06b2ed3a9058ac2f59a5027e99f5 not found: ID does not exist" containerID="bac1e5a58ad503d7f1ae760f5a93d643535b06b2ed3a9058ac2f59a5027e99f5" Jul 16 12:46:25.035561 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.035483 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bac1e5a58ad503d7f1ae760f5a93d643535b06b2ed3a9058ac2f59a5027e99f5"} err="failed to get container status \"bac1e5a58ad503d7f1ae760f5a93d643535b06b2ed3a9058ac2f59a5027e99f5\": rpc error: code = NotFound desc = could not find container \"bac1e5a58ad503d7f1ae760f5a93d643535b06b2ed3a9058ac2f59a5027e99f5\": container with ID starting with bac1e5a58ad503d7f1ae760f5a93d643535b06b2ed3a9058ac2f59a5027e99f5 not found: ID does not exist" Jul 16 12:46:25.057789 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.057766 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-credential-socket\") pod \"89970cf0-95c2-42e9-9b84-6e8b22854227\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " Jul 16 12:46:25.057880 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.057801 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-workload-certs\") pod \"89970cf0-95c2-42e9-9b84-6e8b22854227\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " Jul 16 12:46:25.057880 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.057816 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-workload-socket\") pod \"89970cf0-95c2-42e9-9b84-6e8b22854227\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " Jul 16 12:46:25.057946 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.057879 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-token\") pod \"89970cf0-95c2-42e9-9b84-6e8b22854227\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " Jul 16 12:46:25.057946 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.057900 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-data\") pod \"89970cf0-95c2-42e9-9b84-6e8b22854227\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " Jul 16 12:46:25.057946 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.057940 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89970cf0-95c2-42e9-9b84-6e8b22854227-istiod-ca-cert\") pod \"89970cf0-95c2-42e9-9b84-6e8b22854227\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " Jul 16 12:46:25.058080 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.057985 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5qfk\" (UniqueName: \"kubernetes.io/projected/89970cf0-95c2-42e9-9b84-6e8b22854227-kube-api-access-z5qfk\") pod \"89970cf0-95c2-42e9-9b84-6e8b22854227\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " Jul 16 12:46:25.058080 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.058009 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-envoy\") pod \"89970cf0-95c2-42e9-9b84-6e8b22854227\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " Jul 16 12:46:25.058080 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.058040 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "89970cf0-95c2-42e9-9b84-6e8b22854227" (UID: "89970cf0-95c2-42e9-9b84-6e8b22854227"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:46:25.058080 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.058054 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-podinfo\") pod \"89970cf0-95c2-42e9-9b84-6e8b22854227\" (UID: \"89970cf0-95c2-42e9-9b84-6e8b22854227\") " Jul 16 12:46:25.058279 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.058088 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "89970cf0-95c2-42e9-9b84-6e8b22854227" (UID: "89970cf0-95c2-42e9-9b84-6e8b22854227"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:46:25.058279 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.058193 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "89970cf0-95c2-42e9-9b84-6e8b22854227" (UID: "89970cf0-95c2-42e9-9b84-6e8b22854227"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:46:25.058414 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.058403 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-credential-socket\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:25.058469 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.058425 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-workload-certs\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:25.058469 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.058439 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-workload-socket\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:25.058469 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.058444 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89970cf0-95c2-42e9-9b84-6e8b22854227-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "89970cf0-95c2-42e9-9b84-6e8b22854227" (UID: "89970cf0-95c2-42e9-9b84-6e8b22854227"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:46:25.058640 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.058533 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-data" (OuterVolumeSpecName: "istio-data") pod "89970cf0-95c2-42e9-9b84-6e8b22854227" (UID: "89970cf0-95c2-42e9-9b84-6e8b22854227"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:46:25.060412 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.060387 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-token" (OuterVolumeSpecName: "istio-token") pod "89970cf0-95c2-42e9-9b84-6e8b22854227" (UID: "89970cf0-95c2-42e9-9b84-6e8b22854227"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:46:25.060412 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.060399 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89970cf0-95c2-42e9-9b84-6e8b22854227-kube-api-access-z5qfk" (OuterVolumeSpecName: "kube-api-access-z5qfk") pod "89970cf0-95c2-42e9-9b84-6e8b22854227" (UID: "89970cf0-95c2-42e9-9b84-6e8b22854227"). InnerVolumeSpecName "kube-api-access-z5qfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:46:25.060657 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.060636 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "89970cf0-95c2-42e9-9b84-6e8b22854227" (UID: "89970cf0-95c2-42e9-9b84-6e8b22854227"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 12:46:25.060712 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.060694 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "89970cf0-95c2-42e9-9b84-6e8b22854227" (UID: "89970cf0-95c2-42e9-9b84-6e8b22854227"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:46:25.159069 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.159035 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-token\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:25.159069 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.159061 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-data\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:25.159069 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.159069 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89970cf0-95c2-42e9-9b84-6e8b22854227-istiod-ca-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:25.159332 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.159080 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5qfk\" (UniqueName: \"kubernetes.io/projected/89970cf0-95c2-42e9-9b84-6e8b22854227-kube-api-access-z5qfk\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:25.159332 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.159090 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-envoy\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:25.159332 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.159100 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89970cf0-95c2-42e9-9b84-6e8b22854227-istio-podinfo\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:25.349121 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.349088 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv"] Jul 16 12:46:25.356637 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:25.356612 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-767373bc-openshift-default-7bcc5bcc5-vltbv"] Jul 16 12:46:26.638882 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.638847 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89970cf0-95c2-42e9-9b84-6e8b22854227" path="/var/lib/kubelet/pods/89970cf0-95c2-42e9-9b84-6e8b22854227/volumes" Jul 16 12:46:26.759543 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.759512 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq"] Jul 16 12:46:26.759890 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.759877 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89970cf0-95c2-42e9-9b84-6e8b22854227" containerName="istio-proxy" Jul 16 12:46:26.759941 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.759893 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="89970cf0-95c2-42e9-9b84-6e8b22854227" containerName="istio-proxy" Jul 16 12:46:26.759983 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.759974 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="89970cf0-95c2-42e9-9b84-6e8b22854227" containerName="istio-proxy" Jul 16 12:46:26.764514 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.764489 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.767430 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.767412 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-54d3af5b-gw-openshift-default-dockercfg-426rh\"" Jul 16 12:46:26.779072 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.779042 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq"] Jul 16 12:46:26.874238 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.874200 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istiod-ca-cert\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.874414 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.874250 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-workload-socket\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.874414 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.874272 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-workload-certs\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.874414 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.874332 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-podinfo\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.874414 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.874361 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-token\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.874549 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.874432 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-envoy\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.874549 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.874464 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-data\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.874549 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.874497 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-credential-socket\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.874549 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.874514 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnp7f\" (UniqueName: \"kubernetes.io/projected/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-kube-api-access-rnp7f\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975057 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975018 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istiod-ca-cert\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975057 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975058 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-workload-socket\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975333 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975076 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-workload-certs\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975333 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975096 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-podinfo\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975333 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975129 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-token\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975333 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975175 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-envoy\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975333 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975201 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-data\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975333 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975244 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-credential-socket\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975333 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975268 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnp7f\" (UniqueName: \"kubernetes.io/projected/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-kube-api-access-rnp7f\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975679 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975597 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-workload-certs\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975740 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975714 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-data\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975789 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975743 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-workload-socket\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975789 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975759 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-credential-socket\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.975853 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.975836 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istiod-ca-cert\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.977679 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.977654 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-envoy\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.977790 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.977775 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-podinfo\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.986996 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.986972 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-token\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:26.986996 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:26.986982 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnp7f\" (UniqueName: \"kubernetes.io/projected/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-kube-api-access-rnp7f\") pod \"e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:27.077577 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.077531 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:27.175113 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.175083 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv"] Jul 16 12:46:27.179712 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.179686 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:27.181998 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.181977 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-54d3af5b-serving-cert\"" Jul 16 12:46:27.191255 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.191232 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv"] Jul 16 12:46:27.224568 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.224377 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq"] Jul 16 12:46:27.227340 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:46:27.227258 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7580f52d_56d4_4bc9_8b6e_7d57f34eba9d.slice/crio-ae58644534ec598eefe0fd04462ef2f00c7c62a5fe32055b8a450badc2983c98 WatchSource:0}: Error finding container ae58644534ec598eefe0fd04462ef2f00c7c62a5fe32055b8a450badc2983c98: Status 404 returned error can't find the container with id ae58644534ec598eefe0fd04462ef2f00c7c62a5fe32055b8a450badc2983c98 Jul 16 12:46:27.229325 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.229274 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:46:27.229381 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.229358 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:46:27.229417 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.229386 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:46:27.278756 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.278727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e5bd08-28b8-4a57-b06e-cb601c4f634c-openshift-service-ca\") pod \"maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv\" (UID: \"72e5bd08-28b8-4a57-b06e-cb601c4f634c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:27.278915 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.278870 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sgzf\" (UniqueName: \"kubernetes.io/projected/72e5bd08-28b8-4a57-b06e-cb601c4f634c-kube-api-access-4sgzf\") pod \"maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv\" (UID: \"72e5bd08-28b8-4a57-b06e-cb601c4f634c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:27.279161 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.279065 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e5bd08-28b8-4a57-b06e-cb601c4f634c-maas-api-tls\") pod \"maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv\" (UID: \"72e5bd08-28b8-4a57-b06e-cb601c4f634c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:27.380371 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.380338 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e5bd08-28b8-4a57-b06e-cb601c4f634c-maas-api-tls\") pod \"maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv\" (UID: \"72e5bd08-28b8-4a57-b06e-cb601c4f634c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:27.380534 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.380406 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e5bd08-28b8-4a57-b06e-cb601c4f634c-openshift-service-ca\") pod \"maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv\" (UID: \"72e5bd08-28b8-4a57-b06e-cb601c4f634c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:27.380534 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.380482 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sgzf\" (UniqueName: \"kubernetes.io/projected/72e5bd08-28b8-4a57-b06e-cb601c4f634c-kube-api-access-4sgzf\") pod \"maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv\" (UID: \"72e5bd08-28b8-4a57-b06e-cb601c4f634c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:27.380534 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:46:27.380507 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-serving-cert: secret "maas-api-e2e-migrate-54d3af5b-serving-cert" not found Jul 16 12:46:27.380715 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:46:27.380573 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72e5bd08-28b8-4a57-b06e-cb601c4f634c-maas-api-tls podName:72e5bd08-28b8-4a57-b06e-cb601c4f634c nodeName:}" failed. No retries permitted until 2026-07-16 12:46:27.880556637 +0000 UTC m=+1551.742457859 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/72e5bd08-28b8-4a57-b06e-cb601c4f634c-maas-api-tls") pod "maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" (UID: "72e5bd08-28b8-4a57-b06e-cb601c4f634c") : secret "maas-api-e2e-migrate-54d3af5b-serving-cert" not found Jul 16 12:46:27.381240 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.381206 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e5bd08-28b8-4a57-b06e-cb601c4f634c-openshift-service-ca\") pod \"maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv\" (UID: \"72e5bd08-28b8-4a57-b06e-cb601c4f634c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:27.391406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.391381 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sgzf\" (UniqueName: \"kubernetes.io/projected/72e5bd08-28b8-4a57-b06e-cb601c4f634c-kube-api-access-4sgzf\") pod \"maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv\" (UID: \"72e5bd08-28b8-4a57-b06e-cb601c4f634c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:27.885796 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.885759 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e5bd08-28b8-4a57-b06e-cb601c4f634c-maas-api-tls\") pod \"maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv\" (UID: \"72e5bd08-28b8-4a57-b06e-cb601c4f634c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:27.888467 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:27.888442 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/72e5bd08-28b8-4a57-b06e-cb601c4f634c-maas-api-tls\") pod \"maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv\" (UID: \"72e5bd08-28b8-4a57-b06e-cb601c4f634c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:28.049154 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:28.049114 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" event={"ID":"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d","Type":"ContainerStarted","Data":"a0feafae7cc05282ba09109283055f3466c2cd1dca361f6655516bd1b10e6ec3"} Jul 16 12:46:28.049362 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:28.049171 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" event={"ID":"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d","Type":"ContainerStarted","Data":"ae58644534ec598eefe0fd04462ef2f00c7c62a5fe32055b8a450badc2983c98"} Jul 16 12:46:28.074832 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:28.074763 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" podStartSLOduration=2.074739252 podStartE2EDuration="2.074739252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:46:28.073576239 +0000 UTC m=+1551.935477483" watchObservedRunningTime="2026-07-16 12:46:28.074739252 +0000 UTC m=+1551.936640501" Jul 16 12:46:28.078025 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:28.078000 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:28.083530 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:28.083356 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:28.096457 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:28.096428 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:28.237532 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:28.237505 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv"] Jul 16 12:46:28.239174 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:46:28.239145 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e5bd08_28b8_4a57_b06e_cb601c4f634c.slice/crio-569f3e236fabb79e307b52e8dbd9559450359f200eede5199c6097ae023536dd WatchSource:0}: Error finding container 569f3e236fabb79e307b52e8dbd9559450359f200eede5199c6097ae023536dd: Status 404 returned error can't find the container with id 569f3e236fabb79e307b52e8dbd9559450359f200eede5199c6097ae023536dd Jul 16 12:46:29.056158 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:29.056075 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" event={"ID":"72e5bd08-28b8-4a57-b06e-cb601c4f634c","Type":"ContainerStarted","Data":"584c1de34fa01f09c38c8fe875f5a8432d5d43d748dc87d2deea3933d1494515"} Jul 16 12:46:29.056158 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:29.056125 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" event={"ID":"72e5bd08-28b8-4a57-b06e-cb601c4f634c","Type":"ContainerStarted","Data":"569f3e236fabb79e307b52e8dbd9559450359f200eede5199c6097ae023536dd"} Jul 16 12:46:29.056585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:29.056170 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:29.056585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:29.056338 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:29.057156 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:29.057136 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:29.077310 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:29.077256 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" podStartSLOduration=1.6419508889999999 podStartE2EDuration="2.077243721s" totalImagesPullingTime="435.292832ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:46:29.07437271 +0000 UTC m=+1552.936273956" watchObservedRunningTime="2026-07-16 12:46:29.077243721 +0000 UTC m=+1552.939145001" Jul 16 12:46:32.742138 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:32.742099 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq"] Jul 16 12:46:32.742606 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:32.742304 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" podUID="7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" containerName="istio-proxy" containerID="cri-o://a0feafae7cc05282ba09109283055f3466c2cd1dca361f6655516bd1b10e6ec3" gracePeriod=30 Jul 16 12:46:35.072208 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:35.072170 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv" Jul 16 12:46:39.418500 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.418463 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp"] Jul 16 12:46:39.424723 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.424688 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" Jul 16 12:46:39.426877 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.426856 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-f5f31a77-serving-cert\"" Jul 16 12:46:39.431519 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.431491 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp"] Jul 16 12:46:39.505677 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.505587 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b93a2125-45ed-4486-88a7-d5b738819d6e-openshift-service-ca\") pod \"maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp\" (UID: \"b93a2125-45ed-4486-88a7-d5b738819d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" Jul 16 12:46:39.505677 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.505627 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz2hz\" (UniqueName: \"kubernetes.io/projected/b93a2125-45ed-4486-88a7-d5b738819d6e-kube-api-access-lz2hz\") pod \"maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp\" (UID: \"b93a2125-45ed-4486-88a7-d5b738819d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" Jul 16 12:46:39.505881 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.505765 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93a2125-45ed-4486-88a7-d5b738819d6e-maas-api-tls\") pod \"maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp\" (UID: \"b93a2125-45ed-4486-88a7-d5b738819d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" Jul 16 12:46:39.606831 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.606793 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93a2125-45ed-4486-88a7-d5b738819d6e-maas-api-tls\") pod \"maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp\" (UID: \"b93a2125-45ed-4486-88a7-d5b738819d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" Jul 16 12:46:39.609221 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.606851 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b93a2125-45ed-4486-88a7-d5b738819d6e-openshift-service-ca\") pod \"maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp\" (UID: \"b93a2125-45ed-4486-88a7-d5b738819d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" Jul 16 12:46:39.609221 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.606876 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz2hz\" (UniqueName: \"kubernetes.io/projected/b93a2125-45ed-4486-88a7-d5b738819d6e-kube-api-access-lz2hz\") pod \"maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp\" (UID: \"b93a2125-45ed-4486-88a7-d5b738819d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" Jul 16 12:46:39.609221 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.607692 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b93a2125-45ed-4486-88a7-d5b738819d6e-openshift-service-ca\") pod \"maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp\" (UID: \"b93a2125-45ed-4486-88a7-d5b738819d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" Jul 16 12:46:39.609415 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.609395 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93a2125-45ed-4486-88a7-d5b738819d6e-maas-api-tls\") pod \"maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp\" (UID: \"b93a2125-45ed-4486-88a7-d5b738819d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" Jul 16 12:46:39.615405 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.615384 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz2hz\" (UniqueName: \"kubernetes.io/projected/b93a2125-45ed-4486-88a7-d5b738819d6e-kube-api-access-lz2hz\") pod \"maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp\" (UID: \"b93a2125-45ed-4486-88a7-d5b738819d6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" Jul 16 12:46:39.738498 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.738469 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" Jul 16 12:46:39.890892 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.890868 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp"] Jul 16 12:46:39.891858 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.891840 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:39.891947 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:46:39.891860 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb93a2125_45ed_4486_88a7_d5b738819d6e.slice/crio-6e1a0ad34cf6607f589c86434fb1cbf39a8dfd96e975b97f9d3d36434f97bcba WatchSource:0}: Error finding container 6e1a0ad34cf6607f589c86434fb1cbf39a8dfd96e975b97f9d3d36434f97bcba: Status 404 returned error can't find the container with id 6e1a0ad34cf6607f589c86434fb1cbf39a8dfd96e975b97f9d3d36434f97bcba Jul 16 12:46:39.909423 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.909399 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-workload-socket\") pod \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " Jul 16 12:46:39.909528 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.909436 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnp7f\" (UniqueName: \"kubernetes.io/projected/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-kube-api-access-rnp7f\") pod \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " Jul 16 12:46:39.909528 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.909453 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-data\") pod \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " Jul 16 12:46:39.909528 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.909479 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-podinfo\") pod \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " Jul 16 12:46:39.909528 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.909509 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-credential-socket\") pod \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " Jul 16 12:46:39.909739 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.909557 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-envoy\") pod \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " Jul 16 12:46:39.909739 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.909616 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istiod-ca-cert\") pod \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " Jul 16 12:46:39.909739 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.909653 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-workload-certs\") pod \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " Jul 16 12:46:39.909739 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.909676 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-token\") pod \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\" (UID: \"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d\") " Jul 16 12:46:39.909739 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.909693 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" (UID: "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:46:39.910010 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.909984 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" (UID: "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:46:39.910071 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.910028 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" (UID: "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:46:39.910071 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.910038 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-workload-socket\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:39.910368 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.910250 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" (UID: "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:46:39.910368 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.910328 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-data" (OuterVolumeSpecName: "istio-data") pod "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" (UID: "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:46:39.913806 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.913779 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" (UID: "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 12:46:39.913978 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.913885 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" (UID: "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:46:39.913978 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.913894 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-token" (OuterVolumeSpecName: "istio-token") pod "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" (UID: "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:46:39.913978 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:39.913969 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-kube-api-access-rnp7f" (OuterVolumeSpecName: "kube-api-access-rnp7f") pod "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" (UID: "7580f52d-56d4-4bc9-8b6e-7d57f34eba9d"). InnerVolumeSpecName "kube-api-access-rnp7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:46:40.011510 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.011479 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnp7f\" (UniqueName: \"kubernetes.io/projected/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-kube-api-access-rnp7f\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:40.011510 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.011502 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-data\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:40.011510 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.011512 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-podinfo\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:40.011736 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.011520 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-credential-socket\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:40.011736 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.011529 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-envoy\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:40.011736 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.011537 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istiod-ca-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:40.011736 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.011545 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-workload-certs\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:40.011736 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.011552 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d-istio-token\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:46:40.116197 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.116111 2577 generic.go:356] "Generic (PLEG): container finished" podID="7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" containerID="a0feafae7cc05282ba09109283055f3466c2cd1dca361f6655516bd1b10e6ec3" exitCode=0 Jul 16 12:46:40.116197 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.116154 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" Jul 16 12:46:40.116448 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.116195 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" event={"ID":"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d","Type":"ContainerDied","Data":"a0feafae7cc05282ba09109283055f3466c2cd1dca361f6655516bd1b10e6ec3"} Jul 16 12:46:40.116448 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.116245 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq" event={"ID":"7580f52d-56d4-4bc9-8b6e-7d57f34eba9d","Type":"ContainerDied","Data":"ae58644534ec598eefe0fd04462ef2f00c7c62a5fe32055b8a450badc2983c98"} Jul 16 12:46:40.116448 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.116274 2577 scope.go:117] "RemoveContainer" containerID="a0feafae7cc05282ba09109283055f3466c2cd1dca361f6655516bd1b10e6ec3" Jul 16 12:46:40.117542 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.117515 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" event={"ID":"b93a2125-45ed-4486-88a7-d5b738819d6e","Type":"ContainerStarted","Data":"6e1a0ad34cf6607f589c86434fb1cbf39a8dfd96e975b97f9d3d36434f97bcba"} Jul 16 12:46:40.125902 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.125885 2577 scope.go:117] "RemoveContainer" containerID="a0feafae7cc05282ba09109283055f3466c2cd1dca361f6655516bd1b10e6ec3" Jul 16 12:46:40.126168 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:46:40.126147 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0feafae7cc05282ba09109283055f3466c2cd1dca361f6655516bd1b10e6ec3\": container with ID starting with a0feafae7cc05282ba09109283055f3466c2cd1dca361f6655516bd1b10e6ec3 not found: ID does not exist" containerID="a0feafae7cc05282ba09109283055f3466c2cd1dca361f6655516bd1b10e6ec3" Jul 16 12:46:40.126212 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.126176 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0feafae7cc05282ba09109283055f3466c2cd1dca361f6655516bd1b10e6ec3"} err="failed to get container status \"a0feafae7cc05282ba09109283055f3466c2cd1dca361f6655516bd1b10e6ec3\": rpc error: code = NotFound desc = could not find container \"a0feafae7cc05282ba09109283055f3466c2cd1dca361f6655516bd1b10e6ec3\": container with ID starting with a0feafae7cc05282ba09109283055f3466c2cd1dca361f6655516bd1b10e6ec3 not found: ID does not exist" Jul 16 12:46:40.143863 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.143839 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq"] Jul 16 12:46:40.152960 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.152937 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-54d3af5b-gw-openshift-default-5f5f6b4699-bn7nq"] Jul 16 12:46:40.640126 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:40.640052 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" path="/var/lib/kubelet/pods/7580f52d-56d4-4bc9-8b6e-7d57f34eba9d/volumes" Jul 16 12:46:41.125376 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:41.125334 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" event={"ID":"b93a2125-45ed-4486-88a7-d5b738819d6e","Type":"ContainerStarted","Data":"4abddd60d3190033600c1a301406c17d7634ecd36afc4cb48c2d393d64e573a8"} Jul 16 12:46:41.125563 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:41.125426 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" Jul 16 12:46:41.141949 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:41.141904 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" podStartSLOduration=1.704738639 podStartE2EDuration="2.141890036s" totalImagesPullingTime="437.151397ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:46:41.141571598 +0000 UTC m=+1565.003472871" watchObservedRunningTime="2026-07-16 12:46:41.141890036 +0000 UTC m=+1565.003791321" Jul 16 12:46:47.136447 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:46:47.136420 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp" Jul 16 12:48:29.311233 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.311200 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv"] Jul 16 12:48:29.311673 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.311559 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" containerName="istio-proxy" Jul 16 12:48:29.311673 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.311571 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" containerName="istio-proxy" Jul 16 12:48:29.311673 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.311636 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7580f52d-56d4-4bc9-8b6e-7d57f34eba9d" containerName="istio-proxy" Jul 16 12:48:29.314616 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.314600 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.316948 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.316924 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2e36a3ff-openshift-default-dockercfg-49n4f\"" Jul 16 12:48:29.336396 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.336372 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv"] Jul 16 12:48:29.410884 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.410749 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-credential-socket\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.410884 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.410812 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-envoy\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.411135 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.410894 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-token\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.411135 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.410959 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3f4622-7074-4a65-88d0-571ff80c43c2-istiod-ca-cert\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.411135 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.410996 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-workload-socket\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.411135 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.411046 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-data\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.411135 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.411069 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-podinfo\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.411465 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.411175 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-workload-certs\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.411465 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.411200 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnx7l\" (UniqueName: \"kubernetes.io/projected/9c3f4622-7074-4a65-88d0-571ff80c43c2-kube-api-access-jnx7l\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.512156 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.512127 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-token\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.512374 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.512167 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3f4622-7074-4a65-88d0-571ff80c43c2-istiod-ca-cert\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.512374 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.512189 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-workload-socket\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.512374 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.512224 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-data\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.512374 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.512249 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-podinfo\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.512374 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.512325 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-workload-certs\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.512656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.512454 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnx7l\" (UniqueName: \"kubernetes.io/projected/9c3f4622-7074-4a65-88d0-571ff80c43c2-kube-api-access-jnx7l\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.512656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.512552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-credential-socket\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.512656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.512588 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-envoy\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.512656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.512645 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-workload-socket\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.512867 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.512670 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-data\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.512867 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.512856 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-workload-certs\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.512982 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.512941 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-credential-socket\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.513222 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.513201 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3f4622-7074-4a65-88d0-571ff80c43c2-istiod-ca-cert\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.514817 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.514797 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-podinfo\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.514887 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.514817 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-envoy\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.522023 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.521996 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-token\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.522113 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.522055 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnx7l\" (UniqueName: \"kubernetes.io/projected/9c3f4622-7074-4a65-88d0-571ff80c43c2-kube-api-access-jnx7l\") pod \"e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.624766 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.624690 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:29.763267 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.763231 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv"] Jul 16 12:48:29.765626 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:48:29.765591 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c3f4622_7074_4a65_88d0_571ff80c43c2.slice/crio-5908f6de0cc50eb85ebf899e707901ee4cff29eff2708388288b2dbcef53d384 WatchSource:0}: Error finding container 5908f6de0cc50eb85ebf899e707901ee4cff29eff2708388288b2dbcef53d384: Status 404 returned error can't find the container with id 5908f6de0cc50eb85ebf899e707901ee4cff29eff2708388288b2dbcef53d384 Jul 16 12:48:29.767881 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.767848 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:48:29.767975 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.767908 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:48:29.767975 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:29.767938 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:48:30.672745 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:30.672708 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" event={"ID":"9c3f4622-7074-4a65-88d0-571ff80c43c2","Type":"ContainerStarted","Data":"4f3acb6255239fb20ebc08c85451af7656789da156c2a09ec7e07eb0c79e14e6"} Jul 16 12:48:30.673138 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:30.672759 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" event={"ID":"9c3f4622-7074-4a65-88d0-571ff80c43c2","Type":"ContainerStarted","Data":"5908f6de0cc50eb85ebf899e707901ee4cff29eff2708388288b2dbcef53d384"} Jul 16 12:48:30.693343 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:30.693263 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" podStartSLOduration=1.6932461220000001 podStartE2EDuration="1.693246122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:48:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:48:30.69312289 +0000 UTC m=+1674.555024165" watchObservedRunningTime="2026-07-16 12:48:30.693246122 +0000 UTC m=+1674.555147368" Jul 16 12:48:31.625836 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:31.625797 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:31.631332 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:31.631287 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:31.679241 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:31.679208 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:31.680384 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:31.680364 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:47.541599 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:47.540882 2577 patch_prober.go:28] interesting pod/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": read tcp 10.133.0.2:58680->10.133.0.45:15021: read: connection reset by peer" start-of-body= Jul 16 12:48:47.541599 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:47.540971 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" podUID="9c3f4622-7074-4a65-88d0-571ff80c43c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": read tcp 10.133.0.2:58680->10.133.0.45:15021: read: connection reset by peer" Jul 16 12:48:47.766916 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:47.766832 2577 generic.go:356] "Generic (PLEG): container finished" podID="9c3f4622-7074-4a65-88d0-571ff80c43c2" containerID="4f3acb6255239fb20ebc08c85451af7656789da156c2a09ec7e07eb0c79e14e6" exitCode=0 Jul 16 12:48:47.766916 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:47.766899 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" event={"ID":"9c3f4622-7074-4a65-88d0-571ff80c43c2","Type":"ContainerDied","Data":"4f3acb6255239fb20ebc08c85451af7656789da156c2a09ec7e07eb0c79e14e6"} Jul 16 12:48:47.767355 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:47.767338 2577 scope.go:117] "RemoveContainer" containerID="4f3acb6255239fb20ebc08c85451af7656789da156c2a09ec7e07eb0c79e14e6" Jul 16 12:48:47.768341 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:47.768316 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:48:47.768407 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:47.768381 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:48:47.768444 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:47.768412 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:48:48.778011 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:48.777967 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" event={"ID":"9c3f4622-7074-4a65-88d0-571ff80c43c2","Type":"ContainerStarted","Data":"6f6b788e30122176ccb6b63d7f17cc51c4eb3edf41a561e0134fd4e5cdbf0eef"} Jul 16 12:48:49.624828 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:49.624794 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:49.625134 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:49.625111 2577 patch_prober.go:28] interesting pod/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 16 12:48:49.625195 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:49.625175 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" podUID="9c3f4622-7074-4a65-88d0-571ff80c43c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 16 12:48:50.625055 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:50.625016 2577 patch_prober.go:28] interesting pod/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 16 12:48:50.625521 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:50.625082 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" podUID="9c3f4622-7074-4a65-88d0-571ff80c43c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 16 12:48:51.284319 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:51.284256 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv"] Jul 16 12:48:51.284541 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:51.284511 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" podUID="9c3f4622-7074-4a65-88d0-571ff80c43c2" containerName="istio-proxy" containerID="cri-o://6f6b788e30122176ccb6b63d7f17cc51c4eb3edf41a561e0134fd4e5cdbf0eef" gracePeriod=30 Jul 16 12:48:58.146019 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.145995 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:58.286490 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.286405 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-podinfo\") pod \"9c3f4622-7074-4a65-88d0-571ff80c43c2\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " Jul 16 12:48:58.286490 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.286456 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-workload-certs\") pod \"9c3f4622-7074-4a65-88d0-571ff80c43c2\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " Jul 16 12:48:58.286490 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.286480 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-data\") pod \"9c3f4622-7074-4a65-88d0-571ff80c43c2\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " Jul 16 12:48:58.286750 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.286538 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-token\") pod \"9c3f4622-7074-4a65-88d0-571ff80c43c2\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " Jul 16 12:48:58.286750 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.286561 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-credential-socket\") pod \"9c3f4622-7074-4a65-88d0-571ff80c43c2\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " Jul 16 12:48:58.286750 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.286585 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-envoy\") pod \"9c3f4622-7074-4a65-88d0-571ff80c43c2\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " Jul 16 12:48:58.286750 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.286617 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-workload-socket\") pod \"9c3f4622-7074-4a65-88d0-571ff80c43c2\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " Jul 16 12:48:58.286750 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.286692 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3f4622-7074-4a65-88d0-571ff80c43c2-istiod-ca-cert\") pod \"9c3f4622-7074-4a65-88d0-571ff80c43c2\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " Jul 16 12:48:58.286750 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.286693 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9c3f4622-7074-4a65-88d0-571ff80c43c2" (UID: "9c3f4622-7074-4a65-88d0-571ff80c43c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:48:58.286750 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.286716 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnx7l\" (UniqueName: \"kubernetes.io/projected/9c3f4622-7074-4a65-88d0-571ff80c43c2-kube-api-access-jnx7l\") pod \"9c3f4622-7074-4a65-88d0-571ff80c43c2\" (UID: \"9c3f4622-7074-4a65-88d0-571ff80c43c2\") " Jul 16 12:48:58.287111 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.286856 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9c3f4622-7074-4a65-88d0-571ff80c43c2" (UID: "9c3f4622-7074-4a65-88d0-571ff80c43c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:48:58.287111 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.286916 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9c3f4622-7074-4a65-88d0-571ff80c43c2" (UID: "9c3f4622-7074-4a65-88d0-571ff80c43c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:48:58.287111 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.287011 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-workload-certs\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:48:58.287111 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.287058 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-credential-socket\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:48:58.287111 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.287073 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-workload-socket\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:48:58.287575 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.287169 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c3f4622-7074-4a65-88d0-571ff80c43c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9c3f4622-7074-4a65-88d0-571ff80c43c2" (UID: "9c3f4622-7074-4a65-88d0-571ff80c43c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:48:58.287575 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.287481 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "9c3f4622-7074-4a65-88d0-571ff80c43c2" (UID: "9c3f4622-7074-4a65-88d0-571ff80c43c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:48:58.288985 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.288956 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9c3f4622-7074-4a65-88d0-571ff80c43c2" (UID: "9c3f4622-7074-4a65-88d0-571ff80c43c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:48:58.288985 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.288962 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "9c3f4622-7074-4a65-88d0-571ff80c43c2" (UID: "9c3f4622-7074-4a65-88d0-571ff80c43c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:48:58.289125 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.289018 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c3f4622-7074-4a65-88d0-571ff80c43c2-kube-api-access-jnx7l" (OuterVolumeSpecName: "kube-api-access-jnx7l") pod "9c3f4622-7074-4a65-88d0-571ff80c43c2" (UID: "9c3f4622-7074-4a65-88d0-571ff80c43c2"). InnerVolumeSpecName "kube-api-access-jnx7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:48:58.289125 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.289098 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9c3f4622-7074-4a65-88d0-571ff80c43c2" (UID: "9c3f4622-7074-4a65-88d0-571ff80c43c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 12:48:58.388406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.388364 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-data\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:48:58.388406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.388400 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-token\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:48:58.388406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.388410 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-envoy\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:48:58.388406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.388417 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c3f4622-7074-4a65-88d0-571ff80c43c2-istiod-ca-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:48:58.388406 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.388426 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnx7l\" (UniqueName: \"kubernetes.io/projected/9c3f4622-7074-4a65-88d0-571ff80c43c2-kube-api-access-jnx7l\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:48:58.388702 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.388434 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c3f4622-7074-4a65-88d0-571ff80c43c2-istio-podinfo\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:48:58.835329 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.835277 2577 generic.go:356] "Generic (PLEG): container finished" podID="9c3f4622-7074-4a65-88d0-571ff80c43c2" containerID="6f6b788e30122176ccb6b63d7f17cc51c4eb3edf41a561e0134fd4e5cdbf0eef" exitCode=0 Jul 16 12:48:58.835513 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.835347 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" Jul 16 12:48:58.835513 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.835376 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" event={"ID":"9c3f4622-7074-4a65-88d0-571ff80c43c2","Type":"ContainerDied","Data":"6f6b788e30122176ccb6b63d7f17cc51c4eb3edf41a561e0134fd4e5cdbf0eef"} Jul 16 12:48:58.835513 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.835433 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv" event={"ID":"9c3f4622-7074-4a65-88d0-571ff80c43c2","Type":"ContainerDied","Data":"5908f6de0cc50eb85ebf899e707901ee4cff29eff2708388288b2dbcef53d384"} Jul 16 12:48:58.835513 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.835463 2577 scope.go:117] "RemoveContainer" containerID="6f6b788e30122176ccb6b63d7f17cc51c4eb3edf41a561e0134fd4e5cdbf0eef" Jul 16 12:48:58.844632 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.844609 2577 scope.go:117] "RemoveContainer" containerID="4f3acb6255239fb20ebc08c85451af7656789da156c2a09ec7e07eb0c79e14e6" Jul 16 12:48:58.854634 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.854611 2577 scope.go:117] "RemoveContainer" containerID="6f6b788e30122176ccb6b63d7f17cc51c4eb3edf41a561e0134fd4e5cdbf0eef" Jul 16 12:48:58.855074 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:48:58.855048 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f6b788e30122176ccb6b63d7f17cc51c4eb3edf41a561e0134fd4e5cdbf0eef\": container with ID starting with 6f6b788e30122176ccb6b63d7f17cc51c4eb3edf41a561e0134fd4e5cdbf0eef not found: ID does not exist" containerID="6f6b788e30122176ccb6b63d7f17cc51c4eb3edf41a561e0134fd4e5cdbf0eef" Jul 16 12:48:58.855165 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.855086 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f6b788e30122176ccb6b63d7f17cc51c4eb3edf41a561e0134fd4e5cdbf0eef"} err="failed to get container status \"6f6b788e30122176ccb6b63d7f17cc51c4eb3edf41a561e0134fd4e5cdbf0eef\": rpc error: code = NotFound desc = could not find container \"6f6b788e30122176ccb6b63d7f17cc51c4eb3edf41a561e0134fd4e5cdbf0eef\": container with ID starting with 6f6b788e30122176ccb6b63d7f17cc51c4eb3edf41a561e0134fd4e5cdbf0eef not found: ID does not exist" Jul 16 12:48:58.855165 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.855110 2577 scope.go:117] "RemoveContainer" containerID="4f3acb6255239fb20ebc08c85451af7656789da156c2a09ec7e07eb0c79e14e6" Jul 16 12:48:58.855462 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:48:58.855438 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f3acb6255239fb20ebc08c85451af7656789da156c2a09ec7e07eb0c79e14e6\": container with ID starting with 4f3acb6255239fb20ebc08c85451af7656789da156c2a09ec7e07eb0c79e14e6 not found: ID does not exist" containerID="4f3acb6255239fb20ebc08c85451af7656789da156c2a09ec7e07eb0c79e14e6" Jul 16 12:48:58.855586 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.855470 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f3acb6255239fb20ebc08c85451af7656789da156c2a09ec7e07eb0c79e14e6"} err="failed to get container status \"4f3acb6255239fb20ebc08c85451af7656789da156c2a09ec7e07eb0c79e14e6\": rpc error: code = NotFound desc = could not find container \"4f3acb6255239fb20ebc08c85451af7656789da156c2a09ec7e07eb0c79e14e6\": container with ID starting with 4f3acb6255239fb20ebc08c85451af7656789da156c2a09ec7e07eb0c79e14e6 not found: ID does not exist" Jul 16 12:48:58.855586 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.855539 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv"] Jul 16 12:48:58.857641 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:48:58.857623 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2e36a3ff-openshift-default-f965b9bb7-6vxwv"] Jul 16 12:49:00.639120 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:49:00.639086 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c3f4622-7074-4a65-88d0-571ff80c43c2" path="/var/lib/kubelet/pods/9c3f4622-7074-4a65-88d0-571ff80c43c2/volumes" Jul 16 12:50:06.912647 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:06.912614 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7"] Jul 16 12:50:06.913097 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:06.912955 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c3f4622-7074-4a65-88d0-571ff80c43c2" containerName="istio-proxy" Jul 16 12:50:06.913097 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:06.912966 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c3f4622-7074-4a65-88d0-571ff80c43c2" containerName="istio-proxy" Jul 16 12:50:06.913097 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:06.912974 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c3f4622-7074-4a65-88d0-571ff80c43c2" containerName="istio-proxy" Jul 16 12:50:06.913097 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:06.912980 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c3f4622-7074-4a65-88d0-571ff80c43c2" containerName="istio-proxy" Jul 16 12:50:06.913097 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:06.913041 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c3f4622-7074-4a65-88d0-571ff80c43c2" containerName="istio-proxy" Jul 16 12:50:06.913097 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:06.913049 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c3f4622-7074-4a65-88d0-571ff80c43c2" containerName="istio-proxy" Jul 16 12:50:06.916114 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:06.916096 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:06.918307 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:06.918271 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-825c20-openshift-default-dockercfg-hlnnc\"" Jul 16 12:50:06.925963 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:06.925941 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7"] Jul 16 12:50:07.007648 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.007609 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-istio-envoy\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.007829 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.007653 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-workload-socket\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.007829 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.007683 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29198c24-db1a-49c7-9477-45830ac64725-istio-token\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.007940 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.007818 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-credential-socket\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.007940 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.007872 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-istio-data\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.007940 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.007917 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29198c24-db1a-49c7-9477-45830ac64725-istiod-ca-cert\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.008086 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.007967 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-workload-certs\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.008086 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.007992 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29198c24-db1a-49c7-9477-45830ac64725-istio-podinfo\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.008086 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.008046 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wszzw\" (UniqueName: \"kubernetes.io/projected/29198c24-db1a-49c7-9477-45830ac64725-kube-api-access-wszzw\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.109457 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.109419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-workload-certs\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.109669 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.109462 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29198c24-db1a-49c7-9477-45830ac64725-istio-podinfo\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.109669 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.109507 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wszzw\" (UniqueName: \"kubernetes.io/projected/29198c24-db1a-49c7-9477-45830ac64725-kube-api-access-wszzw\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.109669 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.109545 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-istio-envoy\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.109669 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.109574 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-workload-socket\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.109669 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.109607 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29198c24-db1a-49c7-9477-45830ac64725-istio-token\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.109972 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.109683 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-credential-socket\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.109972 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.109706 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-istio-data\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.109972 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.109745 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29198c24-db1a-49c7-9477-45830ac64725-istiod-ca-cert\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.110153 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.110012 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-workload-socket\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.110153 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.110068 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-credential-socket\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.110153 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.110113 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-workload-certs\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.110338 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.110279 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-istio-data\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.110585 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.110461 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29198c24-db1a-49c7-9477-45830ac64725-istiod-ca-cert\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.112027 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.112008 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-istio-envoy\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.112433 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.112414 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29198c24-db1a-49c7-9477-45830ac64725-istio-podinfo\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.117673 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.117647 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29198c24-db1a-49c7-9477-45830ac64725-istio-token\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.118485 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.118460 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wszzw\" (UniqueName: \"kubernetes.io/projected/29198c24-db1a-49c7-9477-45830ac64725-kube-api-access-wszzw\") pod \"e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.228032 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.227941 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:07.364528 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.364501 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7"] Jul 16 12:50:07.366172 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:50:07.366145 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29198c24_db1a_49c7_9477_45830ac64725.slice/crio-57bc681a86b0f17f0b94bb3c0a7d43077524deb4094e346acd6659275c8c5c25 WatchSource:0}: Error finding container 57bc681a86b0f17f0b94bb3c0a7d43077524deb4094e346acd6659275c8c5c25: Status 404 returned error can't find the container with id 57bc681a86b0f17f0b94bb3c0a7d43077524deb4094e346acd6659275c8c5c25 Jul 16 12:50:07.368375 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.368333 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:50:07.368449 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.368424 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:50:07.368487 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:07.368474 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:50:08.184758 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:08.184718 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" event={"ID":"29198c24-db1a-49c7-9477-45830ac64725","Type":"ContainerStarted","Data":"70be7a9e4ef32e57133e20bed2ea2ea3ff026571e8a21d77cb8056a5530fe280"} Jul 16 12:50:08.185150 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:08.184775 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" event={"ID":"29198c24-db1a-49c7-9477-45830ac64725","Type":"ContainerStarted","Data":"57bc681a86b0f17f0b94bb3c0a7d43077524deb4094e346acd6659275c8c5c25"} Jul 16 12:50:08.207666 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:08.207610 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" podStartSLOduration=2.207591476 podStartE2EDuration="2.207591476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:50:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:50:08.204889688 +0000 UTC m=+1772.066790982" watchObservedRunningTime="2026-07-16 12:50:08.207591476 +0000 UTC m=+1772.069492722" Jul 16 12:50:08.228664 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:08.228638 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:08.233545 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:08.233522 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:09.186937 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.186901 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr"] Jul 16 12:50:09.190632 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.190608 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:09.190782 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.190701 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:09.191445 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.191423 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:09.192760 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.192744 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-825c20-serving-cert\"" Jul 16 12:50:09.205652 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.205626 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr"] Jul 16 12:50:09.230547 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.230517 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr\" (UID: \"e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:09.230726 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.230662 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c-maas-api-tls\") pod \"maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr\" (UID: \"e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:09.230806 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.230764 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7pwj\" (UniqueName: \"kubernetes.io/projected/e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c-kube-api-access-c7pwj\") pod \"maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr\" (UID: \"e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:09.331874 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.331839 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c-maas-api-tls\") pod \"maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr\" (UID: \"e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:09.332070 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.331912 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7pwj\" (UniqueName: \"kubernetes.io/projected/e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c-kube-api-access-c7pwj\") pod \"maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr\" (UID: \"e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:09.332070 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.331947 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr\" (UID: \"e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:09.332070 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:50:09.331983 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-serving-cert: secret "maas-api-e2e-inf-a-825c20-serving-cert" not found Jul 16 12:50:09.332070 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:50:09.332044 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c-maas-api-tls podName:e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c nodeName:}" failed. No retries permitted until 2026-07-16 12:50:09.83202942 +0000 UTC m=+1773.693930642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c-maas-api-tls") pod "maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" (UID: "e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c") : secret "maas-api-e2e-inf-a-825c20-serving-cert" not found Jul 16 12:50:09.332718 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.332696 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c-openshift-service-ca\") pod \"maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr\" (UID: \"e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:09.341189 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.341161 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7pwj\" (UniqueName: \"kubernetes.io/projected/e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c-kube-api-access-c7pwj\") pod \"maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr\" (UID: \"e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:09.837270 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.837224 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c-maas-api-tls\") pod \"maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr\" (UID: \"e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:09.839892 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:09.839861 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c-maas-api-tls\") pod \"maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr\" (UID: \"e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:10.101544 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:10.101448 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:10.247600 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:10.247576 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr"] Jul 16 12:50:10.249372 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:50:10.249348 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode052f5db_e3b2_4f0d_ad80_89ad3d3ad29c.slice/crio-27bc632b514ca65d74317eeaee2f6a54fe45913ce52dd59d0c0e6c3f0b70d266 WatchSource:0}: Error finding container 27bc632b514ca65d74317eeaee2f6a54fe45913ce52dd59d0c0e6c3f0b70d266: Status 404 returned error can't find the container with id 27bc632b514ca65d74317eeaee2f6a54fe45913ce52dd59d0c0e6c3f0b70d266 Jul 16 12:50:10.251277 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:10.251257 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 12:50:11.209677 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:11.209638 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" event={"ID":"e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c","Type":"ContainerStarted","Data":"f1e60bf8d913c6d74c6ea0197a0e51bc202ef29203fbfcb47160a74bcc4a9302"} Jul 16 12:50:11.209862 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:11.209696 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" event={"ID":"e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c","Type":"ContainerStarted","Data":"27bc632b514ca65d74317eeaee2f6a54fe45913ce52dd59d0c0e6c3f0b70d266"} Jul 16 12:50:11.228797 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:11.228749 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" podStartSLOduration=1.423032238 podStartE2EDuration="2.228730777s" totalImagesPullingTime="805.698539ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:50:11.227446394 +0000 UTC m=+1775.089347639" watchObservedRunningTime="2026-07-16 12:50:11.228730777 +0000 UTC m=+1775.090632029" Jul 16 12:50:12.214827 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:12.214796 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:17.311637 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:17.311601 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb"] Jul 16 12:50:17.315825 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:17.315803 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:17.318728 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:17.318369 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-b2b0c9-serving-cert\"" Jul 16 12:50:17.325852 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:17.325824 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb"] Jul 16 12:50:17.404203 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:17.404166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/752c9a3a-b24c-4800-92bb-94f4f55290b3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb\" (UID: \"752c9a3a-b24c-4800-92bb-94f4f55290b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:17.404387 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:17.404224 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/752c9a3a-b24c-4800-92bb-94f4f55290b3-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb\" (UID: \"752c9a3a-b24c-4800-92bb-94f4f55290b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:17.404387 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:17.404315 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqnxr\" (UniqueName: \"kubernetes.io/projected/752c9a3a-b24c-4800-92bb-94f4f55290b3-kube-api-access-rqnxr\") pod \"maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb\" (UID: \"752c9a3a-b24c-4800-92bb-94f4f55290b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:17.505253 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:17.505214 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/752c9a3a-b24c-4800-92bb-94f4f55290b3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb\" (UID: \"752c9a3a-b24c-4800-92bb-94f4f55290b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:17.505457 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:17.505282 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/752c9a3a-b24c-4800-92bb-94f4f55290b3-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb\" (UID: \"752c9a3a-b24c-4800-92bb-94f4f55290b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:17.505457 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:17.505330 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqnxr\" (UniqueName: \"kubernetes.io/projected/752c9a3a-b24c-4800-92bb-94f4f55290b3-kube-api-access-rqnxr\") pod \"maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb\" (UID: \"752c9a3a-b24c-4800-92bb-94f4f55290b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:17.505533 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:50:17.505469 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-serving-cert: secret "maas-api-e2e-inf-b-b2b0c9-serving-cert" not found Jul 16 12:50:17.505568 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:50:17.505540 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752c9a3a-b24c-4800-92bb-94f4f55290b3-maas-api-tls podName:752c9a3a-b24c-4800-92bb-94f4f55290b3 nodeName:}" failed. No retries permitted until 2026-07-16 12:50:18.005523571 +0000 UTC m=+1781.867424793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/752c9a3a-b24c-4800-92bb-94f4f55290b3-maas-api-tls") pod "maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" (UID: "752c9a3a-b24c-4800-92bb-94f4f55290b3") : secret "maas-api-e2e-inf-b-b2b0c9-serving-cert" not found Jul 16 12:50:17.505987 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:17.505970 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/752c9a3a-b24c-4800-92bb-94f4f55290b3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb\" (UID: \"752c9a3a-b24c-4800-92bb-94f4f55290b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:17.515730 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:17.515703 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqnxr\" (UniqueName: \"kubernetes.io/projected/752c9a3a-b24c-4800-92bb-94f4f55290b3-kube-api-access-rqnxr\") pod \"maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb\" (UID: \"752c9a3a-b24c-4800-92bb-94f4f55290b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:18.009942 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:18.009866 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/752c9a3a-b24c-4800-92bb-94f4f55290b3-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb\" (UID: \"752c9a3a-b24c-4800-92bb-94f4f55290b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:18.012653 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:18.012626 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/752c9a3a-b24c-4800-92bb-94f4f55290b3-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb\" (UID: \"752c9a3a-b24c-4800-92bb-94f4f55290b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:18.226997 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:18.226964 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr" Jul 16 12:50:18.228501 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:18.228479 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:18.381220 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:18.381177 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb"] Jul 16 12:50:18.382824 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:50:18.382794 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod752c9a3a_b24c_4800_92bb_94f4f55290b3.slice/crio-79e05123e4fa2af93793d3278cc55d375ab09ff8c89ce8a2b49b9223de1eda83 WatchSource:0}: Error finding container 79e05123e4fa2af93793d3278cc55d375ab09ff8c89ce8a2b49b9223de1eda83: Status 404 returned error can't find the container with id 79e05123e4fa2af93793d3278cc55d375ab09ff8c89ce8a2b49b9223de1eda83 Jul 16 12:50:19.257772 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:19.257735 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" event={"ID":"752c9a3a-b24c-4800-92bb-94f4f55290b3","Type":"ContainerStarted","Data":"139a544b68a8711632d50a9cb5b1d001fac309e5be8a252815d5aee65cd5127a"} Jul 16 12:50:19.258054 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:19.258036 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" event={"ID":"752c9a3a-b24c-4800-92bb-94f4f55290b3","Type":"ContainerStarted","Data":"79e05123e4fa2af93793d3278cc55d375ab09ff8c89ce8a2b49b9223de1eda83"} Jul 16 12:50:19.258991 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:19.258939 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:19.279783 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:19.279727 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" podStartSLOduration=1.863690743 podStartE2EDuration="2.279709144s" totalImagesPullingTime="416.018401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:50:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:50:19.277053846 +0000 UTC m=+1783.138955094" watchObservedRunningTime="2026-07-16 12:50:19.279709144 +0000 UTC m=+1783.141610389" Jul 16 12:50:25.233474 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:25.233427 2577 patch_prober.go:28] interesting pod/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 16 12:50:25.233893 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:25.233503 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" podUID="29198c24-db1a-49c7-9477-45830ac64725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 16 12:50:26.279093 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:26.279064 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb" Jul 16 12:50:26.313333 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:26.313283 2577 generic.go:356] "Generic (PLEG): container finished" podID="29198c24-db1a-49c7-9477-45830ac64725" containerID="70be7a9e4ef32e57133e20bed2ea2ea3ff026571e8a21d77cb8056a5530fe280" exitCode=0 Jul 16 12:50:26.313520 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:26.313334 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" event={"ID":"29198c24-db1a-49c7-9477-45830ac64725","Type":"ContainerDied","Data":"70be7a9e4ef32e57133e20bed2ea2ea3ff026571e8a21d77cb8056a5530fe280"} Jul 16 12:50:26.313859 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:26.313841 2577 scope.go:117] "RemoveContainer" containerID="70be7a9e4ef32e57133e20bed2ea2ea3ff026571e8a21d77cb8056a5530fe280" Jul 16 12:50:26.314973 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:26.314947 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:50:26.315044 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:26.315007 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:50:26.315101 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:26.315043 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:50:27.319806 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:27.319767 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" event={"ID":"29198c24-db1a-49c7-9477-45830ac64725","Type":"ContainerStarted","Data":"f2938fd9d78f59cb83b7f89cdfa938072f58aaeda301433abfdbcf71c0ffe368"} Jul 16 12:50:28.228370 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:28.228336 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:28.228581 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:28.228558 2577 patch_prober.go:28] interesting pod/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 16 12:50:28.228634 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:28.228616 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" podUID="29198c24-db1a-49c7-9477-45830ac64725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 16 12:50:29.228933 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:29.228900 2577 patch_prober.go:28] interesting pod/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 16 12:50:29.229351 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:29.228979 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" podUID="29198c24-db1a-49c7-9477-45830ac64725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 16 12:50:30.229170 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:30.229131 2577 patch_prober.go:28] interesting pod/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 16 12:50:30.229598 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:30.229188 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" podUID="29198c24-db1a-49c7-9477-45830ac64725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 16 12:50:31.229101 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:31.229067 2577 patch_prober.go:28] interesting pod/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 16 12:50:31.229281 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:31.229134 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" podUID="29198c24-db1a-49c7-9477-45830ac64725" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 16 12:50:32.233029 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:32.232996 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:32.233481 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:32.233403 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:32.234036 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:32.234021 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:50:36.655217 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:36.655191 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:50:36.662805 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:50:36.662782 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:52:05.894941 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:05.894861 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7"] Jul 16 12:52:05.895519 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:05.895073 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" podUID="29198c24-db1a-49c7-9477-45830ac64725" containerName="istio-proxy" containerID="cri-o://f2938fd9d78f59cb83b7f89cdfa938072f58aaeda301433abfdbcf71c0ffe368" gracePeriod=30 Jul 16 12:52:11.149582 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.149557 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:52:11.334926 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.334850 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-workload-certs\") pod \"29198c24-db1a-49c7-9477-45830ac64725\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " Jul 16 12:52:11.334926 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.334916 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wszzw\" (UniqueName: \"kubernetes.io/projected/29198c24-db1a-49c7-9477-45830ac64725-kube-api-access-wszzw\") pod \"29198c24-db1a-49c7-9477-45830ac64725\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " Jul 16 12:52:11.335134 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.334954 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29198c24-db1a-49c7-9477-45830ac64725-istio-podinfo\") pod \"29198c24-db1a-49c7-9477-45830ac64725\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " Jul 16 12:52:11.335134 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.334995 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-workload-socket\") pod \"29198c24-db1a-49c7-9477-45830ac64725\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " Jul 16 12:52:11.335134 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.335038 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29198c24-db1a-49c7-9477-45830ac64725-istio-token\") pod \"29198c24-db1a-49c7-9477-45830ac64725\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " Jul 16 12:52:11.335134 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.335061 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-istio-data\") pod \"29198c24-db1a-49c7-9477-45830ac64725\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " Jul 16 12:52:11.335134 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.335083 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29198c24-db1a-49c7-9477-45830ac64725-istiod-ca-cert\") pod \"29198c24-db1a-49c7-9477-45830ac64725\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " Jul 16 12:52:11.335134 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.335124 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-credential-socket\") pod \"29198c24-db1a-49c7-9477-45830ac64725\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " Jul 16 12:52:11.335499 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.335132 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "29198c24-db1a-49c7-9477-45830ac64725" (UID: "29198c24-db1a-49c7-9477-45830ac64725"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:52:11.335499 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.335160 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-istio-envoy\") pod \"29198c24-db1a-49c7-9477-45830ac64725\" (UID: \"29198c24-db1a-49c7-9477-45830ac64725\") " Jul 16 12:52:11.335499 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.335333 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "29198c24-db1a-49c7-9477-45830ac64725" (UID: "29198c24-db1a-49c7-9477-45830ac64725"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:52:11.335664 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.335495 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29198c24-db1a-49c7-9477-45830ac64725-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "29198c24-db1a-49c7-9477-45830ac64725" (UID: "29198c24-db1a-49c7-9477-45830ac64725"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:52:11.335664 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.335511 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-workload-certs\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:11.335664 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.335532 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-workload-socket\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:11.336011 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.335914 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "29198c24-db1a-49c7-9477-45830ac64725" (UID: "29198c24-db1a-49c7-9477-45830ac64725"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:52:11.336166 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.336081 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-istio-data" (OuterVolumeSpecName: "istio-data") pod "29198c24-db1a-49c7-9477-45830ac64725" (UID: "29198c24-db1a-49c7-9477-45830ac64725"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:52:11.337317 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.337273 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29198c24-db1a-49c7-9477-45830ac64725-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "29198c24-db1a-49c7-9477-45830ac64725" (UID: "29198c24-db1a-49c7-9477-45830ac64725"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 12:52:11.337433 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.337406 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29198c24-db1a-49c7-9477-45830ac64725-istio-token" (OuterVolumeSpecName: "istio-token") pod "29198c24-db1a-49c7-9477-45830ac64725" (UID: "29198c24-db1a-49c7-9477-45830ac64725"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:52:11.337433 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.337418 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29198c24-db1a-49c7-9477-45830ac64725-kube-api-access-wszzw" (OuterVolumeSpecName: "kube-api-access-wszzw") pod "29198c24-db1a-49c7-9477-45830ac64725" (UID: "29198c24-db1a-49c7-9477-45830ac64725"). InnerVolumeSpecName "kube-api-access-wszzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:52:11.337523 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.337465 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "29198c24-db1a-49c7-9477-45830ac64725" (UID: "29198c24-db1a-49c7-9477-45830ac64725"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:52:11.436371 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.436342 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wszzw\" (UniqueName: \"kubernetes.io/projected/29198c24-db1a-49c7-9477-45830ac64725-kube-api-access-wszzw\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:11.436371 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.436367 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29198c24-db1a-49c7-9477-45830ac64725-istio-podinfo\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:11.436371 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.436377 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29198c24-db1a-49c7-9477-45830ac64725-istio-token\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:11.436569 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.436385 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-istio-data\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:11.436569 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.436392 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29198c24-db1a-49c7-9477-45830ac64725-istiod-ca-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:11.436569 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.436400 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-credential-socket\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:11.436569 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.436408 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29198c24-db1a-49c7-9477-45830ac64725-istio-envoy\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:11.873987 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.873954 2577 generic.go:356] "Generic (PLEG): container finished" podID="29198c24-db1a-49c7-9477-45830ac64725" containerID="f2938fd9d78f59cb83b7f89cdfa938072f58aaeda301433abfdbcf71c0ffe368" exitCode=0 Jul 16 12:52:11.873987 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.873993 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" event={"ID":"29198c24-db1a-49c7-9477-45830ac64725","Type":"ContainerDied","Data":"f2938fd9d78f59cb83b7f89cdfa938072f58aaeda301433abfdbcf71c0ffe368"} Jul 16 12:52:11.874251 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.874006 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" Jul 16 12:52:11.874251 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.874033 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7" event={"ID":"29198c24-db1a-49c7-9477-45830ac64725","Type":"ContainerDied","Data":"57bc681a86b0f17f0b94bb3c0a7d43077524deb4094e346acd6659275c8c5c25"} Jul 16 12:52:11.874251 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.874054 2577 scope.go:117] "RemoveContainer" containerID="f2938fd9d78f59cb83b7f89cdfa938072f58aaeda301433abfdbcf71c0ffe368" Jul 16 12:52:11.883611 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.883594 2577 scope.go:117] "RemoveContainer" containerID="70be7a9e4ef32e57133e20bed2ea2ea3ff026571e8a21d77cb8056a5530fe280" Jul 16 12:52:11.891356 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.891337 2577 scope.go:117] "RemoveContainer" containerID="f2938fd9d78f59cb83b7f89cdfa938072f58aaeda301433abfdbcf71c0ffe368" Jul 16 12:52:11.891585 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:52:11.891568 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2938fd9d78f59cb83b7f89cdfa938072f58aaeda301433abfdbcf71c0ffe368\": container with ID starting with f2938fd9d78f59cb83b7f89cdfa938072f58aaeda301433abfdbcf71c0ffe368 not found: ID does not exist" containerID="f2938fd9d78f59cb83b7f89cdfa938072f58aaeda301433abfdbcf71c0ffe368" Jul 16 12:52:11.891630 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.891592 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2938fd9d78f59cb83b7f89cdfa938072f58aaeda301433abfdbcf71c0ffe368"} err="failed to get container status \"f2938fd9d78f59cb83b7f89cdfa938072f58aaeda301433abfdbcf71c0ffe368\": rpc error: code = NotFound desc = could not find container \"f2938fd9d78f59cb83b7f89cdfa938072f58aaeda301433abfdbcf71c0ffe368\": container with ID starting with f2938fd9d78f59cb83b7f89cdfa938072f58aaeda301433abfdbcf71c0ffe368 not found: ID does not exist" Jul 16 12:52:11.891630 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.891607 2577 scope.go:117] "RemoveContainer" containerID="70be7a9e4ef32e57133e20bed2ea2ea3ff026571e8a21d77cb8056a5530fe280" Jul 16 12:52:11.891818 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:52:11.891800 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70be7a9e4ef32e57133e20bed2ea2ea3ff026571e8a21d77cb8056a5530fe280\": container with ID starting with 70be7a9e4ef32e57133e20bed2ea2ea3ff026571e8a21d77cb8056a5530fe280 not found: ID does not exist" containerID="70be7a9e4ef32e57133e20bed2ea2ea3ff026571e8a21d77cb8056a5530fe280" Jul 16 12:52:11.891858 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.891825 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70be7a9e4ef32e57133e20bed2ea2ea3ff026571e8a21d77cb8056a5530fe280"} err="failed to get container status \"70be7a9e4ef32e57133e20bed2ea2ea3ff026571e8a21d77cb8056a5530fe280\": rpc error: code = NotFound desc = could not find container \"70be7a9e4ef32e57133e20bed2ea2ea3ff026571e8a21d77cb8056a5530fe280\": container with ID starting with 70be7a9e4ef32e57133e20bed2ea2ea3ff026571e8a21d77cb8056a5530fe280 not found: ID does not exist" Jul 16 12:52:11.897694 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.897670 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7"] Jul 16 12:52:11.903163 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:11.903143 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-825c20-openshift-default-5998d45dcc-49mj7"] Jul 16 12:52:12.638793 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:12.638760 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29198c24-db1a-49c7-9477-45830ac64725" path="/var/lib/kubelet/pods/29198c24-db1a-49c7-9477-45830ac64725/volumes" Jul 16 12:52:13.795455 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.795400 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx"] Jul 16 12:52:13.795929 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.795911 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29198c24-db1a-49c7-9477-45830ac64725" containerName="istio-proxy" Jul 16 12:52:13.795929 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.795931 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="29198c24-db1a-49c7-9477-45830ac64725" containerName="istio-proxy" Jul 16 12:52:13.796030 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.795944 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29198c24-db1a-49c7-9477-45830ac64725" containerName="istio-proxy" Jul 16 12:52:13.796030 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.795950 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="29198c24-db1a-49c7-9477-45830ac64725" containerName="istio-proxy" Jul 16 12:52:13.796030 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.796013 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="29198c24-db1a-49c7-9477-45830ac64725" containerName="istio-proxy" Jul 16 12:52:13.800346 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.800321 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:13.802463 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.802444 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-467ea0-openshift-default-dockercfg-hmg9t\"" Jul 16 12:52:13.813071 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.813049 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx"] Jul 16 12:52:13.959720 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.959691 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-token\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:13.959902 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.959733 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-envoy\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:13.959902 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.959759 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-credential-socket\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:13.959902 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.959777 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-podinfo\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:13.959902 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.959805 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz9h9\" (UniqueName: \"kubernetes.io/projected/6a6fe936-1daa-49a2-9d42-63421ac68775-kube-api-access-vz9h9\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:13.959902 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.959848 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-workload-socket\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:13.959902 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.959883 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a6fe936-1daa-49a2-9d42-63421ac68775-istiod-ca-cert\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:13.959902 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.959905 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-workload-certs\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:13.960140 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:13.959968 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-data\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.061234 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.061163 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-envoy\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.061234 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.061202 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-credential-socket\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.061234 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.061222 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-podinfo\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.061483 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.061334 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz9h9\" (UniqueName: \"kubernetes.io/projected/6a6fe936-1daa-49a2-9d42-63421ac68775-kube-api-access-vz9h9\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.061483 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.061370 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-workload-socket\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.061483 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.061420 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a6fe936-1daa-49a2-9d42-63421ac68775-istiod-ca-cert\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.061483 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.061453 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-workload-certs\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.061685 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.061492 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-data\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.061685 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.061566 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-token\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.061685 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.061654 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-credential-socket\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.061853 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.061750 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-workload-socket\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.061923 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.061900 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-data\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.062051 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.061925 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-workload-certs\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.062198 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.062177 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a6fe936-1daa-49a2-9d42-63421ac68775-istiod-ca-cert\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.063649 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.063631 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-envoy\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.063885 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.063870 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-podinfo\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.069961 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.069935 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-token\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.070115 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.070100 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz9h9\" (UniqueName: \"kubernetes.io/projected/6a6fe936-1daa-49a2-9d42-63421ac68775-kube-api-access-vz9h9\") pod \"e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.109955 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.109932 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:14.247972 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.247945 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx"] Jul 16 12:52:14.249358 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:52:14.249329 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a6fe936_1daa_49a2_9d42_63421ac68775.slice/crio-d661496e690d792ec846e8a78cc9ee1fea9459eefec6e2bc8a386465239b840b WatchSource:0}: Error finding container d661496e690d792ec846e8a78cc9ee1fea9459eefec6e2bc8a386465239b840b: Status 404 returned error can't find the container with id d661496e690d792ec846e8a78cc9ee1fea9459eefec6e2bc8a386465239b840b Jul 16 12:52:14.251419 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.251392 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:52:14.251491 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.251451 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:52:14.251491 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.251480 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:52:14.895582 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.895543 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" event={"ID":"6a6fe936-1daa-49a2-9d42-63421ac68775","Type":"ContainerStarted","Data":"e24a9349a7b40dd6efb7fd04599287780f10f032f861ad47234502a222414375"} Jul 16 12:52:14.895951 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.895606 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" event={"ID":"6a6fe936-1daa-49a2-9d42-63421ac68775","Type":"ContainerStarted","Data":"d661496e690d792ec846e8a78cc9ee1fea9459eefec6e2bc8a386465239b840b"} Jul 16 12:52:14.920619 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:14.920561 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" podStartSLOduration=1.920544006 podStartE2EDuration="1.920544006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:52:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:52:14.919363957 +0000 UTC m=+1898.781265219" watchObservedRunningTime="2026-07-16 12:52:14.920544006 +0000 UTC m=+1898.782445269" Jul 16 12:52:15.110436 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:15.110403 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:15.115510 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:15.115486 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:15.901192 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:15.901159 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:15.902119 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:15.902102 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:30.903045 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:30.903005 2577 patch_prober.go:28] interesting pod/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 16 12:52:30.903470 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:30.903074 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 16 12:52:30.988968 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:30.988938 2577 generic.go:356] "Generic (PLEG): container finished" podID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerID="e24a9349a7b40dd6efb7fd04599287780f10f032f861ad47234502a222414375" exitCode=0 Jul 16 12:52:30.989130 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:30.989019 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" event={"ID":"6a6fe936-1daa-49a2-9d42-63421ac68775","Type":"ContainerDied","Data":"e24a9349a7b40dd6efb7fd04599287780f10f032f861ad47234502a222414375"} Jul 16 12:52:30.989432 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:30.989418 2577 scope.go:117] "RemoveContainer" containerID="e24a9349a7b40dd6efb7fd04599287780f10f032f861ad47234502a222414375" Jul 16 12:52:30.990405 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:30.990382 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:52:30.990479 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:30.990446 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:52:30.990516 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:30.990482 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:52:31.996270 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:31.996236 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" event={"ID":"6a6fe936-1daa-49a2-9d42-63421ac68775","Type":"ContainerStarted","Data":"fd03f80fda2f0eeab07b249319c7650ca8e7f44ef0a58f69dcbac5c1f0b7fd1f"} Jul 16 12:52:32.111117 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:32.111085 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:32.111306 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:32.111268 2577 patch_prober.go:28] interesting pod/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 16 12:52:32.111364 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:32.111334 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 16 12:52:33.111182 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:33.111128 2577 patch_prober.go:28] interesting pod/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 16 12:52:33.111629 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:33.111209 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 16 12:52:34.111144 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:34.111109 2577 patch_prober.go:28] interesting pod/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 16 12:52:34.111342 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:34.111169 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 16 12:52:35.110523 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:35.110488 2577 patch_prober.go:28] interesting pod/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 16 12:52:35.110705 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:35.110546 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 16 12:52:36.114981 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:36.114951 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:36.115495 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:36.115318 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:36.116014 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:36.115992 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:50.380732 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:50.380700 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx"] Jul 16 12:52:50.381164 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:50.380903 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerName="istio-proxy" containerID="cri-o://fd03f80fda2f0eeab07b249319c7650ca8e7f44ef0a58f69dcbac5c1f0b7fd1f" gracePeriod=30 Jul 16 12:52:51.118222 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:51.118186 2577 patch_prober.go:28] interesting pod/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 12:52:51.118429 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:51.118259 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 12:52:55.634574 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.634549 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:55.756972 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.756935 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vz9h9\" (UniqueName: \"kubernetes.io/projected/6a6fe936-1daa-49a2-9d42-63421ac68775-kube-api-access-vz9h9\") pod \"6a6fe936-1daa-49a2-9d42-63421ac68775\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " Jul 16 12:52:55.757183 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.756981 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-token\") pod \"6a6fe936-1daa-49a2-9d42-63421ac68775\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " Jul 16 12:52:55.757183 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.757005 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-workload-certs\") pod \"6a6fe936-1daa-49a2-9d42-63421ac68775\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " Jul 16 12:52:55.757183 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.757034 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-workload-socket\") pod \"6a6fe936-1daa-49a2-9d42-63421ac68775\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " Jul 16 12:52:55.757183 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.757070 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-data\") pod \"6a6fe936-1daa-49a2-9d42-63421ac68775\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " Jul 16 12:52:55.757183 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.757129 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-credential-socket\") pod \"6a6fe936-1daa-49a2-9d42-63421ac68775\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " Jul 16 12:52:55.757183 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.757150 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-envoy\") pod \"6a6fe936-1daa-49a2-9d42-63421ac68775\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " Jul 16 12:52:55.757183 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.757177 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-podinfo\") pod \"6a6fe936-1daa-49a2-9d42-63421ac68775\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " Jul 16 12:52:55.757589 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.757216 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a6fe936-1daa-49a2-9d42-63421ac68775-istiod-ca-cert\") pod \"6a6fe936-1daa-49a2-9d42-63421ac68775\" (UID: \"6a6fe936-1daa-49a2-9d42-63421ac68775\") " Jul 16 12:52:55.757589 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.757377 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6a6fe936-1daa-49a2-9d42-63421ac68775" (UID: "6a6fe936-1daa-49a2-9d42-63421ac68775"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:52:55.757589 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.757467 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6a6fe936-1daa-49a2-9d42-63421ac68775" (UID: "6a6fe936-1daa-49a2-9d42-63421ac68775"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:52:55.757589 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.757572 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-credential-socket\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:55.757589 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.757590 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-workload-certs\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:55.757861 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.757711 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6a6fe936-1daa-49a2-9d42-63421ac68775" (UID: "6a6fe936-1daa-49a2-9d42-63421ac68775"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:52:55.758238 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.758208 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a6fe936-1daa-49a2-9d42-63421ac68775-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6a6fe936-1daa-49a2-9d42-63421ac68775" (UID: "6a6fe936-1daa-49a2-9d42-63421ac68775"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:52:55.758238 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.758216 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-data" (OuterVolumeSpecName: "istio-data") pod "6a6fe936-1daa-49a2-9d42-63421ac68775" (UID: "6a6fe936-1daa-49a2-9d42-63421ac68775"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:52:55.759468 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.759446 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-token" (OuterVolumeSpecName: "istio-token") pod "6a6fe936-1daa-49a2-9d42-63421ac68775" (UID: "6a6fe936-1daa-49a2-9d42-63421ac68775"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:52:55.759468 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.759454 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a6fe936-1daa-49a2-9d42-63421ac68775-kube-api-access-vz9h9" (OuterVolumeSpecName: "kube-api-access-vz9h9") pod "6a6fe936-1daa-49a2-9d42-63421ac68775" (UID: "6a6fe936-1daa-49a2-9d42-63421ac68775"). InnerVolumeSpecName "kube-api-access-vz9h9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:52:55.759602 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.759519 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6a6fe936-1daa-49a2-9d42-63421ac68775" (UID: "6a6fe936-1daa-49a2-9d42-63421ac68775"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:52:55.759845 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.759821 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6a6fe936-1daa-49a2-9d42-63421ac68775" (UID: "6a6fe936-1daa-49a2-9d42-63421ac68775"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 12:52:55.858476 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.858442 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-workload-socket\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:55.858476 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.858472 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-data\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:55.858476 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.858481 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-envoy\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:55.858714 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.858489 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-podinfo\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:55.858714 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.858496 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a6fe936-1daa-49a2-9d42-63421ac68775-istiod-ca-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:55.858714 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.858506 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz9h9\" (UniqueName: \"kubernetes.io/projected/6a6fe936-1daa-49a2-9d42-63421ac68775-kube-api-access-vz9h9\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:55.858714 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:55.858515 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a6fe936-1daa-49a2-9d42-63421ac68775-istio-token\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:52:56.130178 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:56.130095 2577 generic.go:356] "Generic (PLEG): container finished" podID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerID="fd03f80fda2f0eeab07b249319c7650ca8e7f44ef0a58f69dcbac5c1f0b7fd1f" exitCode=0 Jul 16 12:52:56.130178 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:56.130144 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" Jul 16 12:52:56.130421 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:56.130173 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" event={"ID":"6a6fe936-1daa-49a2-9d42-63421ac68775","Type":"ContainerDied","Data":"fd03f80fda2f0eeab07b249319c7650ca8e7f44ef0a58f69dcbac5c1f0b7fd1f"} Jul 16 12:52:56.130421 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:56.130223 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx" event={"ID":"6a6fe936-1daa-49a2-9d42-63421ac68775","Type":"ContainerDied","Data":"d661496e690d792ec846e8a78cc9ee1fea9459eefec6e2bc8a386465239b840b"} Jul 16 12:52:56.130421 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:56.130244 2577 scope.go:117] "RemoveContainer" containerID="fd03f80fda2f0eeab07b249319c7650ca8e7f44ef0a58f69dcbac5c1f0b7fd1f" Jul 16 12:52:56.140980 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:56.140901 2577 scope.go:117] "RemoveContainer" containerID="e24a9349a7b40dd6efb7fd04599287780f10f032f861ad47234502a222414375" Jul 16 12:52:56.149942 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:56.149917 2577 scope.go:117] "RemoveContainer" containerID="fd03f80fda2f0eeab07b249319c7650ca8e7f44ef0a58f69dcbac5c1f0b7fd1f" Jul 16 12:52:56.150278 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:52:56.150249 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd03f80fda2f0eeab07b249319c7650ca8e7f44ef0a58f69dcbac5c1f0b7fd1f\": container with ID starting with fd03f80fda2f0eeab07b249319c7650ca8e7f44ef0a58f69dcbac5c1f0b7fd1f not found: ID does not exist" containerID="fd03f80fda2f0eeab07b249319c7650ca8e7f44ef0a58f69dcbac5c1f0b7fd1f" Jul 16 12:52:56.150408 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:56.150280 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd03f80fda2f0eeab07b249319c7650ca8e7f44ef0a58f69dcbac5c1f0b7fd1f"} err="failed to get container status \"fd03f80fda2f0eeab07b249319c7650ca8e7f44ef0a58f69dcbac5c1f0b7fd1f\": rpc error: code = NotFound desc = could not find container \"fd03f80fda2f0eeab07b249319c7650ca8e7f44ef0a58f69dcbac5c1f0b7fd1f\": container with ID starting with fd03f80fda2f0eeab07b249319c7650ca8e7f44ef0a58f69dcbac5c1f0b7fd1f not found: ID does not exist" Jul 16 12:52:56.150408 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:56.150323 2577 scope.go:117] "RemoveContainer" containerID="e24a9349a7b40dd6efb7fd04599287780f10f032f861ad47234502a222414375" Jul 16 12:52:56.150787 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:52:56.150764 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e24a9349a7b40dd6efb7fd04599287780f10f032f861ad47234502a222414375\": container with ID starting with e24a9349a7b40dd6efb7fd04599287780f10f032f861ad47234502a222414375 not found: ID does not exist" containerID="e24a9349a7b40dd6efb7fd04599287780f10f032f861ad47234502a222414375" Jul 16 12:52:56.150876 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:56.150797 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24a9349a7b40dd6efb7fd04599287780f10f032f861ad47234502a222414375"} err="failed to get container status \"e24a9349a7b40dd6efb7fd04599287780f10f032f861ad47234502a222414375\": rpc error: code = NotFound desc = could not find container \"e24a9349a7b40dd6efb7fd04599287780f10f032f861ad47234502a222414375\": container with ID starting with e24a9349a7b40dd6efb7fd04599287780f10f032f861ad47234502a222414375 not found: ID does not exist" Jul 16 12:52:56.152726 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:56.152707 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx"] Jul 16 12:52:56.156816 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:56.156793 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-467ea0-openshift-default-5f58454b8f-m6svx"] Jul 16 12:52:56.638860 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:52:56.638828 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" path="/var/lib/kubelet/pods/6a6fe936-1daa-49a2-9d42-63421ac68775/volumes" Jul 16 12:53:04.799586 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.799544 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln"] Jul 16 12:53:04.799946 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.799901 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerName="istio-proxy" Jul 16 12:53:04.799946 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.799913 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerName="istio-proxy" Jul 16 12:53:04.800016 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.800000 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerName="istio-proxy" Jul 16 12:53:04.800016 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.800007 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="29198c24-db1a-49c7-9477-45830ac64725" containerName="istio-proxy" Jul 16 12:53:04.800016 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.800013 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerName="istio-proxy" Jul 16 12:53:04.802701 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.802685 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.805366 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.805344 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-989150-openshift-default-dockercfg-twzsb\"" Jul 16 12:53:04.817715 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.817694 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln"] Jul 16 12:53:04.829395 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.829368 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-envoy\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.829520 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.829411 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-podinfo\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.829520 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.829494 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cecd09c-9b7a-497b-a56e-853a1117affc-istiod-ca-cert\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.829608 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.829527 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-credential-socket\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.829656 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.829604 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-workload-certs\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.829761 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.829652 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-token\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.829761 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.829683 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-data\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.829761 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.829736 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvkxg\" (UniqueName: \"kubernetes.io/projected/5cecd09c-9b7a-497b-a56e-853a1117affc-kube-api-access-jvkxg\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.829867 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.829810 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-workload-socket\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.930429 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.930391 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cecd09c-9b7a-497b-a56e-853a1117affc-istiod-ca-cert\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.930626 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.930437 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-credential-socket\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.930626 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.930469 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-workload-certs\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.930626 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.930489 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-token\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.930626 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.930506 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-data\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.930850 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.930619 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvkxg\" (UniqueName: \"kubernetes.io/projected/5cecd09c-9b7a-497b-a56e-853a1117affc-kube-api-access-jvkxg\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.930850 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.930745 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-workload-socket\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.930941 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.930864 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-credential-socket\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.930941 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.930901 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-data\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.930941 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.930931 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-envoy\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.931103 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.930956 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-podinfo\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.931103 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.930962 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-workload-certs\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.931205 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.931148 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-workload-socket\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.931270 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.931248 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cecd09c-9b7a-497b-a56e-853a1117affc-istiod-ca-cert\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.933183 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.933157 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-envoy\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.933424 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.933407 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-podinfo\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.939529 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.939507 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-token\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:04.939612 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:04.939529 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvkxg\" (UniqueName: \"kubernetes.io/projected/5cecd09c-9b7a-497b-a56e-853a1117affc-kube-api-access-jvkxg\") pod \"e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:05.112898 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:05.112818 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:05.248538 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:05.248509 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln"] Jul 16 12:53:05.251162 ip-10-0-139-167 kubenswrapper[2577]: W0716 12:53:05.251134 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cecd09c_9b7a_497b_a56e_853a1117affc.slice/crio-539929361a920a007d14edde4b72853203c7c14570ab33c3af825d8a4c1a0820 WatchSource:0}: Error finding container 539929361a920a007d14edde4b72853203c7c14570ab33c3af825d8a4c1a0820: Status 404 returned error can't find the container with id 539929361a920a007d14edde4b72853203c7c14570ab33c3af825d8a4c1a0820 Jul 16 12:53:05.253270 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:05.253229 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:53:05.253374 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:05.253325 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:53:05.253374 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:05.253358 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:53:06.186852 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:06.186820 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" event={"ID":"5cecd09c-9b7a-497b-a56e-853a1117affc","Type":"ContainerStarted","Data":"5e244a72ae9c82da20edf1b808f015799617bbec7572f6e7a455009decfa8cb8"} Jul 16 12:53:06.187257 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:06.186873 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" event={"ID":"5cecd09c-9b7a-497b-a56e-853a1117affc","Type":"ContainerStarted","Data":"539929361a920a007d14edde4b72853203c7c14570ab33c3af825d8a4c1a0820"} Jul 16 12:53:06.210950 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:06.210892 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" podStartSLOduration=2.210871907 podStartE2EDuration="2.210871907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 12:53:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 12:53:06.206513314 +0000 UTC m=+1950.068414559" watchObservedRunningTime="2026-07-16 12:53:06.210871907 +0000 UTC m=+1950.072773154" Jul 16 12:53:07.113799 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:07.113768 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:07.119086 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:07.119059 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:07.191668 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:07.191633 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:07.192527 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:07.192510 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:23.233467 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:23.233425 2577 patch_prober.go:28] interesting pod/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 16 12:53:23.233884 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:23.233494 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" podUID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 16 12:53:25.305215 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:25.305179 2577 generic.go:356] "Generic (PLEG): container finished" podID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerID="5e244a72ae9c82da20edf1b808f015799617bbec7572f6e7a455009decfa8cb8" exitCode=0 Jul 16 12:53:25.305668 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:25.305256 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" event={"ID":"5cecd09c-9b7a-497b-a56e-853a1117affc","Type":"ContainerDied","Data":"5e244a72ae9c82da20edf1b808f015799617bbec7572f6e7a455009decfa8cb8"} Jul 16 12:53:25.305668 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:25.305658 2577 scope.go:117] "RemoveContainer" containerID="5e244a72ae9c82da20edf1b808f015799617bbec7572f6e7a455009decfa8cb8" Jul 16 12:53:25.306689 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:25.306661 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:53:25.306762 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:25.306745 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:53:25.306809 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:25.306793 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 16 12:53:26.312759 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:26.312726 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" event={"ID":"5cecd09c-9b7a-497b-a56e-853a1117affc","Type":"ContainerStarted","Data":"4718d89dfd0c8eba4674067561ec9f822e1fe51ce58e6602cd8bc940b9b50f9f"} Jul 16 12:53:27.113783 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:27.113747 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:27.113976 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:27.113955 2577 patch_prober.go:28] interesting pod/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 16 12:53:27.114036 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:27.114002 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" podUID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 16 12:53:28.113233 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:28.113201 2577 patch_prober.go:28] interesting pod/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 16 12:53:28.113613 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:28.113258 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" podUID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 16 12:53:29.114039 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:29.114000 2577 patch_prober.go:28] interesting pod/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 16 12:53:29.114471 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:29.114068 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" podUID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 16 12:53:30.113390 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:30.113355 2577 patch_prober.go:28] interesting pod/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 16 12:53:30.113554 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:30.113409 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" podUID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 16 12:53:31.117310 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:31.117268 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:31.117761 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:31.117743 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:53:31.118340 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:53:31.118324 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:55:36.687814 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:55:36.687783 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:55:36.697466 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:55:36.697446 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 12:59:40.684125 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:40.684093 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln"] Jul 16 12:59:40.684619 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:40.684286 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" podUID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerName="istio-proxy" containerID="cri-o://4718d89dfd0c8eba4674067561ec9f822e1fe51ce58e6602cd8bc940b9b50f9f" gracePeriod=30 Jul 16 12:59:45.941530 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:45.941503 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:59:46.003148 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.003073 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-token\") pod \"5cecd09c-9b7a-497b-a56e-853a1117affc\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " Jul 16 12:59:46.003148 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.003107 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cecd09c-9b7a-497b-a56e-853a1117affc-istiod-ca-cert\") pod \"5cecd09c-9b7a-497b-a56e-853a1117affc\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " Jul 16 12:59:46.003148 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.003131 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-data\") pod \"5cecd09c-9b7a-497b-a56e-853a1117affc\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " Jul 16 12:59:46.003419 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.003164 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-credential-socket\") pod \"5cecd09c-9b7a-497b-a56e-853a1117affc\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " Jul 16 12:59:46.003419 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.003196 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-workload-certs\") pod \"5cecd09c-9b7a-497b-a56e-853a1117affc\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " Jul 16 12:59:46.003419 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.003233 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvkxg\" (UniqueName: \"kubernetes.io/projected/5cecd09c-9b7a-497b-a56e-853a1117affc-kube-api-access-jvkxg\") pod \"5cecd09c-9b7a-497b-a56e-853a1117affc\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " Jul 16 12:59:46.003419 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.003276 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-workload-socket\") pod \"5cecd09c-9b7a-497b-a56e-853a1117affc\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " Jul 16 12:59:46.003419 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.003350 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-podinfo\") pod \"5cecd09c-9b7a-497b-a56e-853a1117affc\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " Jul 16 12:59:46.003419 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.003413 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-envoy\") pod \"5cecd09c-9b7a-497b-a56e-853a1117affc\" (UID: \"5cecd09c-9b7a-497b-a56e-853a1117affc\") " Jul 16 12:59:46.003712 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.003602 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cecd09c-9b7a-497b-a56e-853a1117affc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5cecd09c-9b7a-497b-a56e-853a1117affc" (UID: "5cecd09c-9b7a-497b-a56e-853a1117affc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 12:59:46.003770 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.003712 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5cecd09c-9b7a-497b-a56e-853a1117affc" (UID: "5cecd09c-9b7a-497b-a56e-853a1117affc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:59:46.003914 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.003888 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5cecd09c-9b7a-497b-a56e-853a1117affc" (UID: "5cecd09c-9b7a-497b-a56e-853a1117affc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:59:46.004005 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.003945 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5cecd09c-9b7a-497b-a56e-853a1117affc" (UID: "5cecd09c-9b7a-497b-a56e-853a1117affc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:59:46.004366 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.004200 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-data" (OuterVolumeSpecName: "istio-data") pod "5cecd09c-9b7a-497b-a56e-853a1117affc" (UID: "5cecd09c-9b7a-497b-a56e-853a1117affc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:59:46.005793 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.005764 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5cecd09c-9b7a-497b-a56e-853a1117affc" (UID: "5cecd09c-9b7a-497b-a56e-853a1117affc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 12:59:46.005900 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.005792 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5cecd09c-9b7a-497b-a56e-853a1117affc" (UID: "5cecd09c-9b7a-497b-a56e-853a1117affc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 16 12:59:46.006021 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.006003 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-token" (OuterVolumeSpecName: "istio-token") pod "5cecd09c-9b7a-497b-a56e-853a1117affc" (UID: "5cecd09c-9b7a-497b-a56e-853a1117affc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:59:46.006105 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.006088 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cecd09c-9b7a-497b-a56e-853a1117affc-kube-api-access-jvkxg" (OuterVolumeSpecName: "kube-api-access-jvkxg") pod "5cecd09c-9b7a-497b-a56e-853a1117affc" (UID: "5cecd09c-9b7a-497b-a56e-853a1117affc"). InnerVolumeSpecName "kube-api-access-jvkxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 12:59:46.104774 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.104734 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-data\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:59:46.104774 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.104772 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-credential-socket\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:59:46.104774 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.104782 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-workload-certs\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:59:46.105018 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.104793 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvkxg\" (UniqueName: \"kubernetes.io/projected/5cecd09c-9b7a-497b-a56e-853a1117affc-kube-api-access-jvkxg\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:59:46.105018 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.104801 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-workload-socket\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:59:46.105018 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.104814 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-podinfo\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:59:46.105018 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.104821 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-envoy\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:59:46.105018 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.104829 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cecd09c-9b7a-497b-a56e-853a1117affc-istio-token\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:59:46.105018 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.104835 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cecd09c-9b7a-497b-a56e-853a1117affc-istiod-ca-cert\") on node \"ip-10-0-139-167.ec2.internal\" DevicePath \"\"" Jul 16 12:59:46.342747 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.342656 2577 generic.go:356] "Generic (PLEG): container finished" podID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerID="4718d89dfd0c8eba4674067561ec9f822e1fe51ce58e6602cd8bc940b9b50f9f" exitCode=0 Jul 16 12:59:46.342747 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.342704 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" Jul 16 12:59:46.342747 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.342738 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" event={"ID":"5cecd09c-9b7a-497b-a56e-853a1117affc","Type":"ContainerDied","Data":"4718d89dfd0c8eba4674067561ec9f822e1fe51ce58e6602cd8bc940b9b50f9f"} Jul 16 12:59:46.342989 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.342811 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln" event={"ID":"5cecd09c-9b7a-497b-a56e-853a1117affc","Type":"ContainerDied","Data":"539929361a920a007d14edde4b72853203c7c14570ab33c3af825d8a4c1a0820"} Jul 16 12:59:46.342989 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.342836 2577 scope.go:117] "RemoveContainer" containerID="4718d89dfd0c8eba4674067561ec9f822e1fe51ce58e6602cd8bc940b9b50f9f" Jul 16 12:59:46.352415 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.352398 2577 scope.go:117] "RemoveContainer" containerID="5e244a72ae9c82da20edf1b808f015799617bbec7572f6e7a455009decfa8cb8" Jul 16 12:59:46.360466 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.360449 2577 scope.go:117] "RemoveContainer" containerID="4718d89dfd0c8eba4674067561ec9f822e1fe51ce58e6602cd8bc940b9b50f9f" Jul 16 12:59:46.360694 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:59:46.360675 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4718d89dfd0c8eba4674067561ec9f822e1fe51ce58e6602cd8bc940b9b50f9f\": container with ID starting with 4718d89dfd0c8eba4674067561ec9f822e1fe51ce58e6602cd8bc940b9b50f9f not found: ID does not exist" containerID="4718d89dfd0c8eba4674067561ec9f822e1fe51ce58e6602cd8bc940b9b50f9f" Jul 16 12:59:46.360745 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.360703 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4718d89dfd0c8eba4674067561ec9f822e1fe51ce58e6602cd8bc940b9b50f9f"} err="failed to get container status \"4718d89dfd0c8eba4674067561ec9f822e1fe51ce58e6602cd8bc940b9b50f9f\": rpc error: code = NotFound desc = could not find container \"4718d89dfd0c8eba4674067561ec9f822e1fe51ce58e6602cd8bc940b9b50f9f\": container with ID starting with 4718d89dfd0c8eba4674067561ec9f822e1fe51ce58e6602cd8bc940b9b50f9f not found: ID does not exist" Jul 16 12:59:46.360745 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.360720 2577 scope.go:117] "RemoveContainer" containerID="5e244a72ae9c82da20edf1b808f015799617bbec7572f6e7a455009decfa8cb8" Jul 16 12:59:46.360926 ip-10-0-139-167 kubenswrapper[2577]: E0716 12:59:46.360900 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e244a72ae9c82da20edf1b808f015799617bbec7572f6e7a455009decfa8cb8\": container with ID starting with 5e244a72ae9c82da20edf1b808f015799617bbec7572f6e7a455009decfa8cb8 not found: ID does not exist" containerID="5e244a72ae9c82da20edf1b808f015799617bbec7572f6e7a455009decfa8cb8" Jul 16 12:59:46.360973 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.360929 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e244a72ae9c82da20edf1b808f015799617bbec7572f6e7a455009decfa8cb8"} err="failed to get container status \"5e244a72ae9c82da20edf1b808f015799617bbec7572f6e7a455009decfa8cb8\": rpc error: code = NotFound desc = could not find container \"5e244a72ae9c82da20edf1b808f015799617bbec7572f6e7a455009decfa8cb8\": container with ID starting with 5e244a72ae9c82da20edf1b808f015799617bbec7572f6e7a455009decfa8cb8 not found: ID does not exist" Jul 16 12:59:46.367179 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.367157 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln"] Jul 16 12:59:46.372207 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.372188 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-989150-openshift-default-7bcddb9c5f-mvgln"] Jul 16 12:59:46.639507 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:46.639430 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cecd09c-9b7a-497b-a56e-853a1117affc" path="/var/lib/kubelet/pods/5cecd09c-9b7a-497b-a56e-853a1117affc/volumes" Jul 16 12:59:54.668234 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:54.668192 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74b78db6b9-ljb2m_76469e9c-8006-46b4-9a56-e4a02be42384/manager/0.log" Jul 16 12:59:55.189399 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:55.189372 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-87d97db45-g8k6l_f2224488-af18-4b2f-88cf-201c9a2c1c1b/copy-manifests/0.log" Jul 16 12:59:55.200684 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:55.200659 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-87d97db45-g8k6l_f2224488-af18-4b2f-88cf-201c9a2c1c1b/manager/0.log" Jul 16 12:59:55.688130 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:55.688096 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-584d9dd8c9-pnc6q_929d5fab-6f4e-4c32-b721-c5d207b77982/maas-api/0.log" Jul 16 12:59:55.807806 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:55.807779 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-e2e-ait-767373bc-5ff4467c86-lk824_aedee2fb-4228-418a-857b-33e714c2baea/maas-api/0.log" Jul 16 12:59:56.170423 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:56.170397 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-e2e-derive-f5f31a77-8b9965975-tdvwp_b93a2125-45ed-4486-88a7-d5b738819d6e/maas-api/0.log" Jul 16 12:59:56.298147 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:56.298118 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-e2e-inf-a-825c20-7cd7474ffc-684nr_e052f5db-e3b2-4f0d-ad80-89ad3d3ad29c/maas-api/0.log" Jul 16 12:59:56.418078 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:56.418051 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-e2e-inf-b-b2b0c9-56c5d44c4d-d9xwb_752c9a3a-b24c-4800-92bb-94f4f55290b3/maas-api/0.log" Jul 16 12:59:56.537442 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:56.537367 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-e2e-migrate-54d3af5b-766dbc6c9-rkkpv_72e5bd08-28b8-4a57-b06e-cb601c4f634c/maas-api/0.log" Jul 16 12:59:57.144230 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:57.144202 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29736765-7rxbc_3caad597-6c8a-40d5-97e3-4dde3f8b162c/cleanup/0.log" Jul 16 12:59:58.068307 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:58.068272 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj_358e5b93-c06a-4028-978a-3b257fbae74f/util/0.log" Jul 16 12:59:58.075558 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:58.075526 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj_358e5b93-c06a-4028-978a-3b257fbae74f/pull/0.log" Jul 16 12:59:58.082507 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:58.082488 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj_358e5b93-c06a-4028-978a-3b257fbae74f/extract/0.log" Jul 16 12:59:58.199607 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:58.199573 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj_824bb465-eeba-430d-918b-6f69632cbadf/util/0.log" Jul 16 12:59:58.207042 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:58.207018 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj_824bb465-eeba-430d-918b-6f69632cbadf/pull/0.log" Jul 16 12:59:58.214045 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:58.214026 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj_824bb465-eeba-430d-918b-6f69632cbadf/extract/0.log" Jul 16 12:59:58.332336 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:58.332240 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb_041982e5-e1fd-45f3-a105-3a794ef52d18/util/0.log" Jul 16 12:59:58.339945 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:58.339925 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb_041982e5-e1fd-45f3-a105-3a794ef52d18/pull/0.log" Jul 16 12:59:58.348553 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:58.348535 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb_041982e5-e1fd-45f3-a105-3a794ef52d18/extract/0.log" Jul 16 12:59:58.465516 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:58.465492 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg_bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce/extract/0.log" Jul 16 12:59:58.474930 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:58.474908 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg_bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce/util/0.log" Jul 16 12:59:58.482692 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:58.482675 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg_bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce/pull/0.log" Jul 16 12:59:59.856661 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:59.856632 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678nl764_44f90567-cca7-491d-8c6f-a08ce954afb7/istio-proxy/0.log" Jul 16 12:59:59.976500 ip-10-0-139-167 kubenswrapper[2577]: I0716 12:59:59.976473 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2rlnt_9483cf16-7c69-4ace-9ba7-087053569a6a/discovery/0.log" Jul 16 13:00:00.162086 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.162051 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736780-5m58t"] Jul 16 13:00:00.162588 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.162569 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerName="istio-proxy" Jul 16 13:00:00.162588 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.162590 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerName="istio-proxy" Jul 16 13:00:00.162739 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.162598 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerName="istio-proxy" Jul 16 13:00:00.162739 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.162603 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerName="istio-proxy" Jul 16 13:00:00.162739 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.162616 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerName="istio-proxy" Jul 16 13:00:00.162739 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.162621 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a6fe936-1daa-49a2-9d42-63421ac68775" containerName="istio-proxy" Jul 16 13:00:00.162739 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.162680 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerName="istio-proxy" Jul 16 13:00:00.162739 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.162687 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cecd09c-9b7a-497b-a56e-853a1117affc" containerName="istio-proxy" Jul 16 13:00:00.167401 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.167382 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736780-5m58t" Jul 16 13:00:00.169572 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.169554 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zjpqc\"" Jul 16 13:00:00.177054 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.177013 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736780-5m58t"] Jul 16 13:00:00.179066 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.179048 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736780-5m58t" Jul 16 13:00:00.188781 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.188754 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-989150-29736780-gn5v5"] Jul 16 13:00:00.193098 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.193058 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-989150-29736780-gn5v5" Jul 16 13:00:00.193963 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.193941 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-54d3af5b-29736780-6cnqf"] Jul 16 13:00:00.197186 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.197166 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-54d3af5b-29736780-6cnqf" Jul 16 13:00:00.197276 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.197218 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-197335-29736780-28mks"] Jul 16 13:00:00.201174 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.201155 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-197335-29736780-28mks" Jul 16 13:00:00.209663 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.209634 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-989150-29736780-gn5v5"] Jul 16 13:00:00.213104 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.212750 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-54d3af5b-29736780-6cnqf"] Jul 16 13:00:00.215507 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.215488 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-293c75-29736780-r6cbl"] Jul 16 13:00:00.219582 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.219563 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-293c75-29736780-r6cbl" Jul 16 13:00:00.222481 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.222445 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-989150-29736780-gn5v5" Jul 16 13:00:00.229176 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.227134 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-197335-29736780-28mks"] Jul 16 13:00:00.231893 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.231834 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-293c75-29736780-r6cbl"] Jul 16 13:00:00.237204 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.236633 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-54d3af5b-29736780-6cnqf" Jul 16 13:00:00.282014 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.281986 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-197335-29736780-28mks" Jul 16 13:00:00.289945 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.289918 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-293c75-29736780-r6cbl" Jul 16 13:00:00.365020 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.364973 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fh8dn_46882479-a6f2-48fe-8886-a81bbf1d101e/istio-proxy/1.log" Jul 16 13:00:00.388674 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.388044 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736780-5m58t"] Jul 16 13:00:00.395772 ip-10-0-139-167 kubenswrapper[2577]: W0716 13:00:00.395633 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c197612_a04c_4c78_89f7_a5005acacd5b.slice/crio-c49a185e21e565e0cf1f7b96848ad6ff2930abc66e2a141a9a7ae2ef0f0aea03 WatchSource:0}: Error finding container c49a185e21e565e0cf1f7b96848ad6ff2930abc66e2a141a9a7ae2ef0f0aea03: Status 404 returned error can't find the container with id c49a185e21e565e0cf1f7b96848ad6ff2930abc66e2a141a9a7ae2ef0f0aea03 Jul 16 13:00:00.423935 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.423816 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736780-5m58t" event={"ID":"6c197612-a04c-4c78-89f7-a5005acacd5b","Type":"ContainerStarted","Data":"c49a185e21e565e0cf1f7b96848ad6ff2930abc66e2a141a9a7ae2ef0f0aea03"} Jul 16 13:00:00.450770 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.450742 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-989150-29736780-gn5v5"] Jul 16 13:00:00.450944 ip-10-0-139-167 kubenswrapper[2577]: W0716 13:00:00.450776 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9754bc_09e2_4a96_94cc_740ada158a92.slice/crio-b3f94264cb8e1d1064f21016199d36c52e76ac1874036355fedc7550f02890b4 WatchSource:0}: Error finding container b3f94264cb8e1d1064f21016199d36c52e76ac1874036355fedc7550f02890b4: Status 404 returned error can't find the container with id b3f94264cb8e1d1064f21016199d36c52e76ac1874036355fedc7550f02890b4 Jul 16 13:00:00.482360 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.482104 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-54d3af5b-29736780-6cnqf"] Jul 16 13:00:00.497659 ip-10-0-139-167 kubenswrapper[2577]: W0716 13:00:00.497632 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceacbb16_10a0_400b_9db2_1f7caf96c6dc.slice/crio-cfe5f627c144b8750b5fe13d07dc804339efe517c2e00ca5589784fab2b5c009 WatchSource:0}: Error finding container cfe5f627c144b8750b5fe13d07dc804339efe517c2e00ca5589784fab2b5c009: Status 404 returned error can't find the container with id cfe5f627c144b8750b5fe13d07dc804339efe517c2e00ca5589784fab2b5c009 Jul 16 13:00:00.512149 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.512121 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-197335-29736780-28mks"] Jul 16 13:00:00.513684 ip-10-0-139-167 kubenswrapper[2577]: W0716 13:00:00.513648 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb7143bc_db6e_4ff4_9faf_f990bbfa7dad.slice/crio-ecb6d9028c3de1f7a210ab3e0e9860d0c816a0d5ce5648642bcdfaa5fb205d5f WatchSource:0}: Error finding container ecb6d9028c3de1f7a210ab3e0e9860d0c816a0d5ce5648642bcdfaa5fb205d5f: Status 404 returned error can't find the container with id ecb6d9028c3de1f7a210ab3e0e9860d0c816a0d5ce5648642bcdfaa5fb205d5f Jul 16 13:00:00.531933 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.531913 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-293c75-29736780-r6cbl"] Jul 16 13:00:00.536272 ip-10-0-139-167 kubenswrapper[2577]: W0716 13:00:00.536245 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf172e159_ffda_41a3_b8b7_7bb003923257.slice/crio-0138bf159bc6c1ef64d3f305183d959082bf581cdc4699d0c365664ca873d67b WatchSource:0}: Error finding container 0138bf159bc6c1ef64d3f305183d959082bf581cdc4699d0c365664ca873d67b: Status 404 returned error can't find the container with id 0138bf159bc6c1ef64d3f305183d959082bf581cdc4699d0c365664ca873d67b Jul 16 13:00:00.614402 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:00.614353 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-6p265_8218c232-5e0c-445d-aa1e-b7bff65ecb6d/payload-processing/0.log" Jul 16 13:00:01.430784 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.430747 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-989150-29736780-gn5v5" event={"ID":"cb9754bc-09e2-4a96-94cc-740ada158a92","Type":"ContainerStarted","Data":"46875644b636556bf9f126e6b0dc52227597dc639268e9360408fb0254384218"} Jul 16 13:00:01.431210 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.430801 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-989150-29736780-gn5v5" event={"ID":"cb9754bc-09e2-4a96-94cc-740ada158a92","Type":"ContainerStarted","Data":"b3f94264cb8e1d1064f21016199d36c52e76ac1874036355fedc7550f02890b4"} Jul 16 13:00:01.432356 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.432326 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736780-5m58t" event={"ID":"6c197612-a04c-4c78-89f7-a5005acacd5b","Type":"ContainerStarted","Data":"10a46365d09946d524a13f131ef8413a5a4d356ff3fe299ea76bbd67af0f9f80"} Jul 16 13:00:01.433682 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.433665 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-293c75-29736780-r6cbl" event={"ID":"f172e159-ffda-41a3-b8b7-7bb003923257","Type":"ContainerStarted","Data":"104f61b7cfe5b94d4d9d671e1d3d74062727910703bb9ecdb99caf2d76823f2d"} Jul 16 13:00:01.433794 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.433697 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-293c75-29736780-r6cbl" event={"ID":"f172e159-ffda-41a3-b8b7-7bb003923257","Type":"ContainerStarted","Data":"0138bf159bc6c1ef64d3f305183d959082bf581cdc4699d0c365664ca873d67b"} Jul 16 13:00:01.435188 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.435158 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-54d3af5b-29736780-6cnqf" event={"ID":"ceacbb16-10a0-400b-9db2-1f7caf96c6dc","Type":"ContainerStarted","Data":"055f40acafd2d6f62e75d36942689847d6b00e15c33ae9b5adb1513e8ad878f2"} Jul 16 13:00:01.435311 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.435209 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-54d3af5b-29736780-6cnqf" event={"ID":"ceacbb16-10a0-400b-9db2-1f7caf96c6dc","Type":"ContainerStarted","Data":"cfe5f627c144b8750b5fe13d07dc804339efe517c2e00ca5589784fab2b5c009"} Jul 16 13:00:01.436546 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.436529 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-197335-29736780-28mks" event={"ID":"cb7143bc-db6e-4ff4-9faf-f990bbfa7dad","Type":"ContainerStarted","Data":"3d902b5a957a3665e5ce7866ac03c728e7a9d144bb867cfadd838747e013bd07"} Jul 16 13:00:01.436614 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.436561 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-197335-29736780-28mks" event={"ID":"cb7143bc-db6e-4ff4-9faf-f990bbfa7dad","Type":"ContainerStarted","Data":"ecb6d9028c3de1f7a210ab3e0e9860d0c816a0d5ce5648642bcdfaa5fb205d5f"} Jul 16 13:00:01.444547 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.444522 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db27s9s_6e2f730c-9a5f-47c3-bc71-8f0ca4c6bedc/main/0.log" Jul 16 13:00:01.448118 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.448066 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-989150-29736780-gn5v5" podStartSLOduration=1.448050255 podStartE2EDuration="1.448050255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:00:01.44701813 +0000 UTC m=+2365.308919377" watchObservedRunningTime="2026-07-16 13:00:01.448050255 +0000 UTC m=+2365.309951500" Jul 16 13:00:01.465805 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.465762 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-293c75-29736780-r6cbl" podStartSLOduration=1.465747407 podStartE2EDuration="1.465747407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:00:01.464453449 +0000 UTC m=+2365.326354728" watchObservedRunningTime="2026-07-16 13:00:01.465747407 +0000 UTC m=+2365.327648630" Jul 16 13:00:01.481803 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.481760 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-54d3af5b-29736780-6cnqf" podStartSLOduration=1.481744864 podStartE2EDuration="1.481744864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:00:01.480382269 +0000 UTC m=+2365.342283535" watchObservedRunningTime="2026-07-16 13:00:01.481744864 +0000 UTC m=+2365.343646109" Jul 16 13:00:01.497264 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.497214 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736780-5m58t" podStartSLOduration=1.497198608 podStartE2EDuration="1.497198608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:00:01.496490522 +0000 UTC m=+2365.358391766" watchObservedRunningTime="2026-07-16 13:00:01.497198608 +0000 UTC m=+2365.359099853" Jul 16 13:00:01.514835 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:01.514780 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-197335-29736780-28mks" podStartSLOduration=1.514760109 podStartE2EDuration="1.514760109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:00:01.514146101 +0000 UTC m=+2365.376047348" watchObservedRunningTime="2026-07-16 13:00:01.514760109 +0000 UTC m=+2365.376661355" Jul 16 13:00:02.444072 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:02.444032 2577 generic.go:356] "Generic (PLEG): container finished" podID="cb9754bc-09e2-4a96-94cc-740ada158a92" containerID="46875644b636556bf9f126e6b0dc52227597dc639268e9360408fb0254384218" exitCode=0 Jul 16 13:00:02.444545 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:02.444122 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-989150-29736780-gn5v5" event={"ID":"cb9754bc-09e2-4a96-94cc-740ada158a92","Type":"ContainerDied","Data":"46875644b636556bf9f126e6b0dc52227597dc639268e9360408fb0254384218"} Jul 16 13:00:02.445565 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:02.445487 2577 generic.go:356] "Generic (PLEG): container finished" podID="6c197612-a04c-4c78-89f7-a5005acacd5b" containerID="10a46365d09946d524a13f131ef8413a5a4d356ff3fe299ea76bbd67af0f9f80" exitCode=0 Jul 16 13:00:02.445565 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:02.445553 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736780-5m58t" event={"ID":"6c197612-a04c-4c78-89f7-a5005acacd5b","Type":"ContainerDied","Data":"10a46365d09946d524a13f131ef8413a5a4d356ff3fe299ea76bbd67af0f9f80"} Jul 16 13:00:02.446857 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:02.446836 2577 generic.go:356] "Generic (PLEG): container finished" podID="f172e159-ffda-41a3-b8b7-7bb003923257" containerID="104f61b7cfe5b94d4d9d671e1d3d74062727910703bb9ecdb99caf2d76823f2d" exitCode=0 Jul 16 13:00:02.446973 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:02.446900 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-293c75-29736780-r6cbl" event={"ID":"f172e159-ffda-41a3-b8b7-7bb003923257","Type":"ContainerDied","Data":"104f61b7cfe5b94d4d9d671e1d3d74062727910703bb9ecdb99caf2d76823f2d"} Jul 16 13:00:02.448335 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:02.448305 2577 generic.go:356] "Generic (PLEG): container finished" podID="ceacbb16-10a0-400b-9db2-1f7caf96c6dc" containerID="055f40acafd2d6f62e75d36942689847d6b00e15c33ae9b5adb1513e8ad878f2" exitCode=0 Jul 16 13:00:02.448421 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:02.448349 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-54d3af5b-29736780-6cnqf" event={"ID":"ceacbb16-10a0-400b-9db2-1f7caf96c6dc","Type":"ContainerDied","Data":"055f40acafd2d6f62e75d36942689847d6b00e15c33ae9b5adb1513e8ad878f2"} Jul 16 13:00:02.449796 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:02.449779 2577 generic.go:356] "Generic (PLEG): container finished" podID="cb7143bc-db6e-4ff4-9faf-f990bbfa7dad" containerID="3d902b5a957a3665e5ce7866ac03c728e7a9d144bb867cfadd838747e013bd07" exitCode=0 Jul 16 13:00:02.449870 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:02.449810 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-197335-29736780-28mks" event={"ID":"cb7143bc-db6e-4ff4-9faf-f990bbfa7dad","Type":"ContainerDied","Data":"3d902b5a957a3665e5ce7866ac03c728e7a9d144bb867cfadd838747e013bd07"} Jul 16 13:00:03.610061 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:03.610035 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-197335-29736780-28mks" Jul 16 13:00:03.684857 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:03.684511 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-293c75-29736780-r6cbl" Jul 16 13:00:03.697941 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:03.697909 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-54d3af5b-29736780-6cnqf" Jul 16 13:00:03.713361 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:03.713280 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736780-5m58t" Jul 16 13:00:03.716821 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:03.716796 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-989150-29736780-gn5v5" Jul 16 13:00:04.461429 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.461400 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-197335-29736780-28mks" Jul 16 13:00:04.461633 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.461442 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-197335-29736780-28mks" event={"ID":"cb7143bc-db6e-4ff4-9faf-f990bbfa7dad","Type":"ContainerDied","Data":"ecb6d9028c3de1f7a210ab3e0e9860d0c816a0d5ce5648642bcdfaa5fb205d5f"} Jul 16 13:00:04.461633 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.461495 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ecb6d9028c3de1f7a210ab3e0e9860d0c816a0d5ce5648642bcdfaa5fb205d5f" Jul 16 13:00:04.462757 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.462727 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-989150-29736780-gn5v5" Jul 16 13:00:04.462757 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.462737 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-989150-29736780-gn5v5" event={"ID":"cb9754bc-09e2-4a96-94cc-740ada158a92","Type":"ContainerDied","Data":"b3f94264cb8e1d1064f21016199d36c52e76ac1874036355fedc7550f02890b4"} Jul 16 13:00:04.462945 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.462774 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3f94264cb8e1d1064f21016199d36c52e76ac1874036355fedc7550f02890b4" Jul 16 13:00:04.463962 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.463945 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736780-5m58t" Jul 16 13:00:04.464097 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.463960 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736780-5m58t" event={"ID":"6c197612-a04c-4c78-89f7-a5005acacd5b","Type":"ContainerDied","Data":"c49a185e21e565e0cf1f7b96848ad6ff2930abc66e2a141a9a7ae2ef0f0aea03"} Jul 16 13:00:04.464097 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.463999 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c49a185e21e565e0cf1f7b96848ad6ff2930abc66e2a141a9a7ae2ef0f0aea03" Jul 16 13:00:04.465444 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.465426 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-293c75-29736780-r6cbl" Jul 16 13:00:04.465547 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.465439 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-293c75-29736780-r6cbl" event={"ID":"f172e159-ffda-41a3-b8b7-7bb003923257","Type":"ContainerDied","Data":"0138bf159bc6c1ef64d3f305183d959082bf581cdc4699d0c365664ca873d67b"} Jul 16 13:00:04.465547 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.465480 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0138bf159bc6c1ef64d3f305183d959082bf581cdc4699d0c365664ca873d67b" Jul 16 13:00:04.466900 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.466873 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-54d3af5b-29736780-6cnqf" event={"ID":"ceacbb16-10a0-400b-9db2-1f7caf96c6dc","Type":"ContainerDied","Data":"cfe5f627c144b8750b5fe13d07dc804339efe517c2e00ca5589784fab2b5c009"} Jul 16 13:00:04.466900 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.466891 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-migrate-54d3af5b-29736780-6cnqf" Jul 16 13:00:04.467068 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:04.466918 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfe5f627c144b8750b5fe13d07dc804339efe517c2e00ca5589784fab2b5c009" Jul 16 13:00:10.023198 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:10.023169 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wjdcb_4a3e09d3-661a-4a7f-8741-f39758ea7303/global-pull-secret-syncer/0.log" Jul 16 13:00:10.157949 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:10.157915 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sw9kn_7233e4f2-8303-45b4-8870-3ab003f5e5d8/konnectivity-agent/0.log" Jul 16 13:00:10.232811 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:10.232780 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-167.ec2.internal_1bb2e9c0b1e73e2a5eceaddc294c2341/haproxy/0.log" Jul 16 13:00:14.176082 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:14.176054 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj_358e5b93-c06a-4028-978a-3b257fbae74f/extract/0.log" Jul 16 13:00:14.207249 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:14.207223 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj_358e5b93-c06a-4028-978a-3b257fbae74f/util/0.log" Jul 16 13:00:14.235529 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:14.235512 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759brnjj_358e5b93-c06a-4028-978a-3b257fbae74f/pull/0.log" Jul 16 13:00:14.302769 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:14.302746 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj_824bb465-eeba-430d-918b-6f69632cbadf/extract/0.log" Jul 16 13:00:14.329284 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:14.329258 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj_824bb465-eeba-430d-918b-6f69632cbadf/util/0.log" Jul 16 13:00:14.353811 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:14.353791 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ftjmj_824bb465-eeba-430d-918b-6f69632cbadf/pull/0.log" Jul 16 13:00:14.382285 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:14.382264 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb_041982e5-e1fd-45f3-a105-3a794ef52d18/extract/0.log" Jul 16 13:00:14.408639 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:14.408622 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb_041982e5-e1fd-45f3-a105-3a794ef52d18/util/0.log" Jul 16 13:00:14.432776 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:14.432725 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wchmb_041982e5-e1fd-45f3-a105-3a794ef52d18/pull/0.log" Jul 16 13:00:14.461925 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:14.461908 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg_bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce/extract/0.log" Jul 16 13:00:14.486760 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:14.486737 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg_bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce/util/0.log" Jul 16 13:00:14.510764 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:14.510747 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m5ccg_bfd32bd2-ecd8-4b0a-8ec9-d473c3000dce/pull/0.log" Jul 16 13:00:17.487555 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:17.487528 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-655886bb64-7s7t7_dc78434d-b99a-44d8-9d74-4d1b3e3ba1f7/metrics-server/0.log" Jul 16 13:00:17.524180 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:17.524156 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-6k4gp_a3c84aee-058d-4745-bf8e-cdb49a140ad1/monitoring-plugin/0.log" Jul 16 13:00:17.557979 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:17.557954 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hg7qx_029311a2-84cd-4e64-81c5-65c0b73d9aad/node-exporter/0.log" Jul 16 13:00:17.584096 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:17.584072 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hg7qx_029311a2-84cd-4e64-81c5-65c0b73d9aad/kube-rbac-proxy/0.log" Jul 16 13:00:17.614806 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:17.614780 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hg7qx_029311a2-84cd-4e64-81c5-65c0b73d9aad/init-textfile/0.log" Jul 16 13:00:18.313716 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:18.313687 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86446784b8-qlxst_0fb8710e-ff74-4545-b298-654f7a9752a2/thanos-query/0.log" Jul 16 13:00:18.337960 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:18.337930 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86446784b8-qlxst_0fb8710e-ff74-4545-b298-654f7a9752a2/kube-rbac-proxy-web/0.log" Jul 16 13:00:18.364928 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:18.364904 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86446784b8-qlxst_0fb8710e-ff74-4545-b298-654f7a9752a2/kube-rbac-proxy/0.log" Jul 16 13:00:18.396006 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:18.395979 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86446784b8-qlxst_0fb8710e-ff74-4545-b298-654f7a9752a2/prom-label-proxy/0.log" Jul 16 13:00:18.420983 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:18.420963 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86446784b8-qlxst_0fb8710e-ff74-4545-b298-654f7a9752a2/kube-rbac-proxy-rules/0.log" Jul 16 13:00:18.447074 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:18.447041 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-86446784b8-qlxst_0fb8710e-ff74-4545-b298-654f7a9752a2/kube-rbac-proxy-metrics/0.log" Jul 16 13:00:19.665249 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665221 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn"] Jul 16 13:00:19.665656 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665568 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb7143bc-db6e-4ff4-9faf-f990bbfa7dad" containerName="cleanup" Jul 16 13:00:19.665656 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665579 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb7143bc-db6e-4ff4-9faf-f990bbfa7dad" containerName="cleanup" Jul 16 13:00:19.665656 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665589 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceacbb16-10a0-400b-9db2-1f7caf96c6dc" containerName="cleanup" Jul 16 13:00:19.665656 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665594 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceacbb16-10a0-400b-9db2-1f7caf96c6dc" containerName="cleanup" Jul 16 13:00:19.665656 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665600 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f172e159-ffda-41a3-b8b7-7bb003923257" containerName="cleanup" Jul 16 13:00:19.665656 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665605 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f172e159-ffda-41a3-b8b7-7bb003923257" containerName="cleanup" Jul 16 13:00:19.665656 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665616 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb9754bc-09e2-4a96-94cc-740ada158a92" containerName="cleanup" Jul 16 13:00:19.665656 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665622 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb9754bc-09e2-4a96-94cc-740ada158a92" containerName="cleanup" Jul 16 13:00:19.665656 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665634 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c197612-a04c-4c78-89f7-a5005acacd5b" containerName="cleanup" Jul 16 13:00:19.665656 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665639 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c197612-a04c-4c78-89f7-a5005acacd5b" containerName="cleanup" Jul 16 13:00:19.665987 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665702 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c197612-a04c-4c78-89f7-a5005acacd5b" containerName="cleanup" Jul 16 13:00:19.665987 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665709 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f172e159-ffda-41a3-b8b7-7bb003923257" containerName="cleanup" Jul 16 13:00:19.665987 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665715 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb7143bc-db6e-4ff4-9faf-f990bbfa7dad" containerName="cleanup" Jul 16 13:00:19.665987 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665725 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceacbb16-10a0-400b-9db2-1f7caf96c6dc" containerName="cleanup" Jul 16 13:00:19.665987 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.665730 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb9754bc-09e2-4a96-94cc-740ada158a92" containerName="cleanup" Jul 16 13:00:19.668855 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.668839 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.670910 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.670888 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ndv85\"/\"kube-root-ca.crt\"" Jul 16 13:00:19.671024 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.670926 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ndv85\"/\"default-dockercfg-7mnpt\"" Jul 16 13:00:19.671766 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.671752 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ndv85\"/\"openshift-service-ca.crt\"" Jul 16 13:00:19.680405 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.680386 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn"] Jul 16 13:00:19.812671 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.812640 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ebba8311-7124-48b2-85a9-06a8135bef95-sys\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.812671 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.812672 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ebba8311-7124-48b2-85a9-06a8135bef95-proc\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.812863 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.812703 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws527\" (UniqueName: \"kubernetes.io/projected/ebba8311-7124-48b2-85a9-06a8135bef95-kube-api-access-ws527\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.812863 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.812756 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ebba8311-7124-48b2-85a9-06a8135bef95-lib-modules\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.812863 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.812830 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ebba8311-7124-48b2-85a9-06a8135bef95-podres\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.913844 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.913805 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ebba8311-7124-48b2-85a9-06a8135bef95-podres\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.913981 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.913859 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ebba8311-7124-48b2-85a9-06a8135bef95-sys\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.913981 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.913878 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ebba8311-7124-48b2-85a9-06a8135bef95-proc\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.913981 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.913906 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws527\" (UniqueName: \"kubernetes.io/projected/ebba8311-7124-48b2-85a9-06a8135bef95-kube-api-access-ws527\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.913981 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.913922 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ebba8311-7124-48b2-85a9-06a8135bef95-lib-modules\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.913981 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.913946 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ebba8311-7124-48b2-85a9-06a8135bef95-sys\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.914160 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.913986 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ebba8311-7124-48b2-85a9-06a8135bef95-podres\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.914160 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.914016 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ebba8311-7124-48b2-85a9-06a8135bef95-proc\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.914160 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.914062 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ebba8311-7124-48b2-85a9-06a8135bef95-lib-modules\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.921704 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.921644 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws527\" (UniqueName: \"kubernetes.io/projected/ebba8311-7124-48b2-85a9-06a8135bef95-kube-api-access-ws527\") pod \"perf-node-gather-daemonset-xp7wn\" (UID: \"ebba8311-7124-48b2-85a9-06a8135bef95\") " pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:19.978157 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:19.978133 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:20.116009 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:20.115986 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn"] Jul 16 13:00:20.117707 ip-10-0-139-167 kubenswrapper[2577]: W0716 13:00:20.117679 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podebba8311_7124_48b2_85a9_06a8135bef95.slice/crio-d93a9d851fed8915242372498e4cb4a91b13206debc4084d47496f00e4621d17 WatchSource:0}: Error finding container d93a9d851fed8915242372498e4cb4a91b13206debc4084d47496f00e4621d17: Status 404 returned error can't find the container with id d93a9d851fed8915242372498e4cb4a91b13206debc4084d47496f00e4621d17 Jul 16 13:00:20.552501 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:20.552415 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" event={"ID":"ebba8311-7124-48b2-85a9-06a8135bef95","Type":"ContainerStarted","Data":"a94968bc70ba451c8b45ab3d6d5e45ec7485c3d20faff7556e3b05e872333bbb"} Jul 16 13:00:20.552501 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:20.552463 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" event={"ID":"ebba8311-7124-48b2-85a9-06a8135bef95","Type":"ContainerStarted","Data":"d93a9d851fed8915242372498e4cb4a91b13206debc4084d47496f00e4621d17"} Jul 16 13:00:20.552691 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:20.552609 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:20.569382 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:20.569332 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" podStartSLOduration=1.5693147010000001 podStartE2EDuration="1.569314701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 13:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 13:00:20.567948459 +0000 UTC m=+2384.429849718" watchObservedRunningTime="2026-07-16 13:00:20.569314701 +0000 UTC m=+2384.431215938" Jul 16 13:00:22.562365 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:22.562334 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xgspb_86b7ee17-6296-45e8-8f98-f6ca03f75c73/dns/0.log" Jul 16 13:00:22.586106 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:22.586080 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xgspb_86b7ee17-6296-45e8-8f98-f6ca03f75c73/kube-rbac-proxy/0.log" Jul 16 13:00:22.652944 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:22.652922 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lxhnm_926f2417-fb83-42f8-abee-55e41d925420/dns-node-resolver/0.log" Jul 16 13:00:23.235506 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:23.235475 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-75f6d5c9cd-ssrzn_f36b38c2-0c06-4df7-8d87-3c574fdeaf31/registry/0.log" Jul 16 13:00:23.279172 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:23.279145 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mw9kk_d7d7182b-abdd-4b23-9f34-2b5bb3b86d2f/node-ca/0.log" Jul 16 13:00:24.254940 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:24.254907 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678nl764_44f90567-cca7-491d-8c6f-a08ce954afb7/istio-proxy/0.log" Jul 16 13:00:24.350390 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:24.350358 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2rlnt_9483cf16-7c69-4ace-9ba7-087053569a6a/discovery/0.log" Jul 16 13:00:24.428991 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:24.428962 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fh8dn_46882479-a6f2-48fe-8886-a81bbf1d101e/istio-proxy/0.log" Jul 16 13:00:24.456198 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:24.456168 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fh8dn_46882479-a6f2-48fe-8886-a81bbf1d101e/istio-proxy/1.log" Jul 16 13:00:24.525563 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:24.525488 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-6p265_8218c232-5e0c-445d-aa1e-b7bff65ecb6d/payload-processing/0.log" Jul 16 13:00:25.219708 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:25.219677 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hzjdc_b8ea5963-0f56-4b84-a76e-c53646bf650d/serve-healthcheck-canary/0.log" Jul 16 13:00:25.886705 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:25.886677 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-99pq4_f283e409-c6e8-4dde-9bbb-4190973d2321/kube-rbac-proxy/0.log" Jul 16 13:00:25.913877 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:25.913851 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-99pq4_f283e409-c6e8-4dde-9bbb-4190973d2321/exporter/0.log" Jul 16 13:00:25.938188 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:25.938163 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-99pq4_f283e409-c6e8-4dde-9bbb-4190973d2321/extractor/0.log" Jul 16 13:00:26.568997 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:26.568972 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ndv85/perf-node-gather-daemonset-xp7wn" Jul 16 13:00:28.457626 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:28.457593 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74b78db6b9-ljb2m_76469e9c-8006-46b4-9a56-e4a02be42384/manager/0.log" Jul 16 13:00:28.742656 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:28.742575 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-87d97db45-g8k6l_f2224488-af18-4b2f-88cf-201c9a2c1c1b/manager/0.log" Jul 16 13:00:28.768352 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:28.768327 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-87d97db45-g8k6l_f2224488-af18-4b2f-88cf-201c9a2c1c1b/copy-manifests/0.log" Jul 16 13:00:29.207697 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:29.207671 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678nl764_44f90567-cca7-491d-8c6f-a08ce954afb7/istio-proxy/0.log" Jul 16 13:00:29.298755 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:29.298724 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2rlnt_9483cf16-7c69-4ace-9ba7-087053569a6a/discovery/0.log" Jul 16 13:00:29.350312 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:29.350264 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fh8dn_46882479-a6f2-48fe-8886-a81bbf1d101e/istio-proxy/0.log" Jul 16 13:00:29.380092 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:29.380065 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fh8dn_46882479-a6f2-48fe-8886-a81bbf1d101e/istio-proxy/1.log" Jul 16 13:00:29.430580 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:29.430553 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-6p265_8218c232-5e0c-445d-aa1e-b7bff65ecb6d/payload-processing/0.log" Jul 16 13:00:30.219895 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:30.219859 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-698fdd4469-vzv4c_827f5bf7-5ca8-4c59-9303-5556bb580e83/manager/0.log" Jul 16 13:00:30.251757 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:30.251727 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rx9jj_d85e0849-773f-4f4a-a2b1-4cf787b34452/openshift-lws-operator/0.log" Jul 16 13:00:30.833199 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:30.833116 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678nl764_44f90567-cca7-491d-8c6f-a08ce954afb7/istio-proxy/0.log" Jul 16 13:00:30.927274 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:30.927244 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2rlnt_9483cf16-7c69-4ace-9ba7-087053569a6a/discovery/0.log" Jul 16 13:00:30.977658 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:30.977628 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fh8dn_46882479-a6f2-48fe-8886-a81bbf1d101e/istio-proxy/0.log" Jul 16 13:00:31.005502 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:31.005473 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-fh8dn_46882479-a6f2-48fe-8886-a81bbf1d101e/istio-proxy/1.log" Jul 16 13:00:31.055078 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:31.055049 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-6p265_8218c232-5e0c-445d-aa1e-b7bff65ecb6d/payload-processing/0.log" Jul 16 13:00:36.723028 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:36.723000 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 13:00:36.732975 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:36.732956 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7cwqf_13b833aa-75f5-490c-8df2-b9d2247c61d6/ovn-acl-logging/0.log" Jul 16 13:00:37.043522 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:37.043480 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kxg9t_5280509a-3dd9-492d-9eae-02e5c64d56e4/kube-multus-additional-cni-plugins/0.log" Jul 16 13:00:37.069661 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:37.069590 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kxg9t_5280509a-3dd9-492d-9eae-02e5c64d56e4/egress-router-binary-copy/0.log" Jul 16 13:00:37.099668 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:37.099643 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kxg9t_5280509a-3dd9-492d-9eae-02e5c64d56e4/cni-plugins/0.log" Jul 16 13:00:37.124017 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:37.123993 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kxg9t_5280509a-3dd9-492d-9eae-02e5c64d56e4/bond-cni-plugin/0.log" Jul 16 13:00:37.152434 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:37.152406 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kxg9t_5280509a-3dd9-492d-9eae-02e5c64d56e4/routeoverride-cni/0.log" Jul 16 13:00:37.186782 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:37.186756 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kxg9t_5280509a-3dd9-492d-9eae-02e5c64d56e4/whereabouts-cni-bincopy/0.log" Jul 16 13:00:37.224227 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:37.224193 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kxg9t_5280509a-3dd9-492d-9eae-02e5c64d56e4/whereabouts-cni/0.log" Jul 16 13:00:37.453469 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:37.453440 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jqf5n_95931072-3e08-422c-aff5-1e519977e11c/kube-multus/0.log" Jul 16 13:00:37.521769 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:37.521739 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-98bqc_bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e/network-metrics-daemon/0.log" Jul 16 13:00:37.547957 ip-10-0-139-167 kubenswrapper[2577]: I0716 13:00:37.547929 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-98bqc_bc6fd46f-f94f-4f9f-8aca-89a63ab2a44e/kube-rbac-proxy/0.log"