Jul 21 06:17:49.952542 ip-10-0-140-159 systemd[1]: Starting Kubernetes Kubelet... Jul 21 06:17:50.293870 ip-10-0-140-159 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 06:17:50.293870 ip-10-0-140-159 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 06:17:50.293870 ip-10-0-140-159 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 06:17:50.293870 ip-10-0-140-159 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 06:17:50.293870 ip-10-0-140-159 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 06:17:50.295530 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.295442 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 06:17:50.299486 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299471 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 06:17:50.299486 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299485 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299489 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299492 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299496 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299499 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299501 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299505 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299508 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299510 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299513 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299516 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299518 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299520 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299523 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299526 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299528 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299531 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299535 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299539 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 06:17:50.299553 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299542 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299544 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299547 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299550 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299553 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299556 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299558 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299561 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299564 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299566 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299569 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299571 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299574 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299576 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299579 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299581 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299584 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299587 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299589 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299592 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 06:17:50.300002 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299595 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299597 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299600 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299602 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299604 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299607 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299609 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299611 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299614 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299616 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299618 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299621 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299623 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299626 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299630 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299633 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299635 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299638 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299640 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299643 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 06:17:50.300511 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299646 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299650 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299653 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299655 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299657 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299660 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299662 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299665 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299667 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299670 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299672 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299674 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299677 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299681 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299684 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299686 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299689 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299692 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299695 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299697 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 06:17:50.301001 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299700 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299702 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299705 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299707 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299709 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299712 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299716 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299718 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299721 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299723 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299726 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299729 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299732 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299734 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299737 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299741 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299744 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299747 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.299750 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 06:17:50.301498 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300433 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300439 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300442 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300445 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300448 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300451 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300454 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300457 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300460 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300463 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300465 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300468 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300471 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300475 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300478 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300480 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300483 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300486 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300488 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 06:17:50.301954 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300492 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300495 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300497 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300500 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300503 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300505 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300508 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300510 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300513 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300516 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300519 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300522 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300524 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300527 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300529 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300532 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300534 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300537 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300540 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 06:17:50.302430 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300544 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300547 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300550 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300553 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300556 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300558 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300561 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300564 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300566 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300569 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300571 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300574 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300576 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300579 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300581 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300584 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300586 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300589 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300591 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300593 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 06:17:50.302881 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300596 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300598 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300601 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300603 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300606 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300608 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300610 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300613 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300615 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300617 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300620 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300622 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300626 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300628 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300630 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300633 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300636 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300640 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300643 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300646 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 06:17:50.303394 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300649 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300651 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300654 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300657 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300659 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300662 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300664 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300667 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300669 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300672 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300674 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300676 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300679 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300681 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300684 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300686 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300689 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300691 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300693 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300696 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 06:17:50.303887 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.300698 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301065 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301071 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301074 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301078 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301080 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301083 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301086 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301089 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301092 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301094 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301097 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301099 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301102 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301104 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301107 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301109 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301114 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301116 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 06:17:50.304373 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301119 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301122 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301124 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301127 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301130 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301132 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301135 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301137 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301140 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301142 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301145 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301147 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301149 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301152 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301154 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301156 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301159 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301161 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301164 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301166 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 06:17:50.304874 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301170 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301174 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301188 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301190 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301193 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301196 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301198 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301200 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301203 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301205 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301210 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301226 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301228 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301231 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301233 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301236 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301238 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301241 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301243 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301247 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 06:17:50.305454 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301251 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301254 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301257 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301260 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301263 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301265 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301268 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301270 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301273 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301275 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301278 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301280 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301283 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301285 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301288 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301291 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301293 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301296 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301299 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301302 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 06:17:50.305990 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301305 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301308 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301312 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301315 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301318 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301320 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301323 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301326 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301329 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301332 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301335 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301337 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301340 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301342 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301345 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301347 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301349 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301352 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301354 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301357 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 06:17:50.306544 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.301359 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301423 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301431 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301436 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301441 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301445 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301448 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301452 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301457 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301460 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301463 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301466 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301469 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301472 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301476 2563 flags.go:64] FLAG: --cgroup-root="" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301479 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301482 2563 flags.go:64] FLAG: --client-ca-file="" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301485 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301488 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301492 2563 flags.go:64] FLAG: --cluster-domain="" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301495 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301498 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301501 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301504 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 06:17:50.307084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301508 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301511 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301514 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301517 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301521 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301523 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301527 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301530 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301532 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301536 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301539 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301541 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301544 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301547 2563 flags.go:64] FLAG: --enable-server="true" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301550 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301554 2563 flags.go:64] FLAG: --event-burst="100" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301557 2563 flags.go:64] FLAG: --event-qps="50" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301560 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301563 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301565 2563 flags.go:64] FLAG: --eviction-hard="" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301569 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301572 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301576 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301578 2563 flags.go:64] FLAG: --eviction-soft="" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301581 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 06:17:50.307853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301584 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301587 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301590 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301592 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301595 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301598 2563 flags.go:64] FLAG: --feature-gates="" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301601 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301604 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301607 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301610 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301613 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301616 2563 flags.go:64] FLAG: --help="false" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301619 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301621 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301624 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301627 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301630 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301633 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301636 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301638 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301641 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301644 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301647 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301650 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 06:17:50.308612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301653 2563 flags.go:64] FLAG: --kube-reserved="" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301656 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301658 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301661 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301664 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301668 2563 flags.go:64] FLAG: --lock-file="" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301671 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301674 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301677 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301682 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301684 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301687 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301690 2563 flags.go:64] FLAG: --logging-format="text" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301692 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301695 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301698 2563 flags.go:64] FLAG: --manifest-url="" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301701 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301705 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301708 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301712 2563 flags.go:64] FLAG: --max-pods="110" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301714 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301718 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301720 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301723 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301726 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 06:17:50.309545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301729 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301731 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301737 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301742 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301745 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301748 2563 flags.go:64] FLAG: --pod-cidr="" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301750 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301755 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301759 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301762 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301764 2563 flags.go:64] FLAG: --port="10250" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301767 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301770 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fbc5e0dce9faf604" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301774 2563 flags.go:64] FLAG: --qos-reserved="" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301777 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301780 2563 flags.go:64] FLAG: --register-node="true" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301783 2563 flags.go:64] FLAG: --register-with-taints="" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301786 2563 flags.go:64] FLAG: --registry-burst="10" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301789 2563 flags.go:64] FLAG: --registry-qps="5" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301792 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301794 2563 flags.go:64] FLAG: --reserved-memory="" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301797 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301801 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301803 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301806 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 06:17:50.310161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301809 2563 flags.go:64] FLAG: --runonce="false" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301812 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301815 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301817 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301820 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301823 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301825 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301828 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301831 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301834 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301838 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301841 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301844 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301847 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301850 2563 flags.go:64] FLAG: --system-cgroups="" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301852 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301858 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301861 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301863 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301867 2563 flags.go:64] FLAG: --tls-min-version="" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301871 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301874 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301877 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301879 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301882 2563 flags.go:64] FLAG: --v="2" Jul 21 06:17:50.310803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301886 2563 flags.go:64] FLAG: --version="false" Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301895 2563 flags.go:64] FLAG: --vmodule="" Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301898 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.301901 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302013 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302017 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302020 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302022 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302026 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302029 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302031 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302034 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302036 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302039 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302041 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302044 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302046 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302050 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302053 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302055 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 06:17:50.311428 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302057 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302060 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302062 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302065 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302067 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302070 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302072 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302076 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302078 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302081 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302083 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302085 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302088 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302090 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302093 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302095 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302097 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302100 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302102 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302104 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 06:17:50.311915 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302107 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302111 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302114 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302118 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302120 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302124 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302127 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302129 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302132 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302136 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302138 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302141 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302143 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302146 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302148 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302150 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302153 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302155 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302158 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 06:17:50.312422 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302161 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302163 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302166 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302168 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302171 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302173 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302175 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302178 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302180 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302183 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302186 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302189 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302192 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302195 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302197 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302200 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302202 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302205 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302207 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 06:17:50.312865 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302209 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302230 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302235 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302240 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302244 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302249 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302252 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302254 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302257 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302259 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302262 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302264 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302268 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302271 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302273 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302276 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302278 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302281 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302283 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302286 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 06:17:50.313405 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302288 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302290 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302293 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302296 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.302298 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.302975 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.310632 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.310647 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.311311 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.311319 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311374 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311380 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311384 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311388 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 06:17:50.313882 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311392 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311395 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311398 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311401 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311404 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311407 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311410 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311413 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311416 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311418 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311421 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311424 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311426 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311429 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311432 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311434 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311437 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311440 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311442 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311445 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 06:17:50.314247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311447 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311450 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311452 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311456 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311458 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311461 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311464 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311466 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311469 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311472 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311474 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311477 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311479 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311482 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311485 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311487 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311490 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311492 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311494 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311497 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 06:17:50.314730 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311499 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311502 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311505 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311508 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311510 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311513 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311515 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311518 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311520 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311522 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311525 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311527 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311547 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311550 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311553 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311556 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311559 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311562 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311565 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311567 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 06:17:50.315207 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311570 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311572 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311575 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311577 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311580 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311582 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311584 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311588 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311591 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311593 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311596 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311599 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311601 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311603 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311606 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311609 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311612 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311615 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311617 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 06:17:50.315714 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311620 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311622 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311624 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311627 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311629 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311632 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311634 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311636 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311638 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311641 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311644 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311646 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311648 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311651 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311653 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 06:17:50.316170 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311656 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.311661 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311774 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311779 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311782 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311786 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311788 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311791 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311794 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311797 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311799 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311802 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311804 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311807 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311810 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 06:17:50.316584 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311813 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311815 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311817 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311820 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311822 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311825 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311827 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311829 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311832 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311834 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311837 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311839 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311842 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311844 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311847 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311849 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311852 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311855 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311858 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 06:17:50.316951 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311861 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311864 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311867 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311870 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311872 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311874 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311877 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311879 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311882 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311884 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311887 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311889 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311891 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311895 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311897 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311899 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311901 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311903 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311906 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311908 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 06:17:50.317558 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311910 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311913 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311915 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311918 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311921 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311924 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311928 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311931 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311934 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311936 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311939 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311941 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311943 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311946 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311948 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311950 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311953 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311955 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311957 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 06:17:50.318023 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311960 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311962 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311964 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311967 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311969 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311971 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311974 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311976 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311979 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311981 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311983 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311986 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311988 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311991 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311993 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311995 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.311997 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.312000 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.312002 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.312005 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 06:17:50.318508 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.312007 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 06:17:50.318977 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.312010 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 06:17:50.318977 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.312013 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 06:17:50.318977 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.312017 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 06:17:50.318977 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.312019 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 06:17:50.318977 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.312022 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 06:17:50.318977 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.312025 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 06:17:50.318977 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:50.312027 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 06:17:50.318977 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.312032 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 06:17:50.318977 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.312134 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 06:17:50.318977 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.315322 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 06:17:50.318977 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.316336 2563 server.go:1019] "Starting client certificate rotation" Jul 21 06:17:50.318977 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.316436 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 06:17:50.318977 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.316493 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 06:17:50.319327 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.318461 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 06:17:50.323718 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.323704 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 06:17:50.332696 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.332678 2563 log.go:25] "Validated CRI v1 runtime API" Jul 21 06:17:50.335098 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.335081 2563 log.go:25] "Validated CRI v1 image API" Jul 21 06:17:50.335990 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.335978 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 06:17:50.338324 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.338304 2563 fs.go:135] Filesystem UUIDs: map[63ac0651-7c20-4736-be89-5fc057806d0f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d1f96be8-0020-435c-b814-a5a13ab019fb:/dev/nvme0n1p4] Jul 21 06:17:50.338405 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.338324 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 06:17:50.343649 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.343516 2563 manager.go:216] Machine: {Timestamp:2026-07-21 06:17:50.342685914 +0000 UTC m=+0.288324563 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097533 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f27ed1b3eae62d8f7c8cbfbf99fd8 SystemUUID:ec2f27ed-1b3e-ae62-d8f7-c8cbfbf99fd8 BootID:c62d2bca-5394-4902-b87d-f97cae445263 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:62:9a:38:17:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:62:9a:38:17:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:56:60:a5:c7:bf Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 06:17:50.343649 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.343637 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 06:17:50.343766 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.343708 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 06:17:50.344259 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.344244 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 06:17:50.345741 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.345720 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 06:17:50.345891 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.345745 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-159.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 06:17:50.346974 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.346963 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 06:17:50.347013 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.346977 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 06:17:50.347013 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.346984 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 06:17:50.347612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.347601 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 06:17:50.348593 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.348582 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 06:17:50.350884 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.350873 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 21 06:17:50.351007 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.350998 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 06:17:50.351048 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.351026 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 21 06:17:50.351048 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.351035 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 06:17:50.351048 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.351046 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 06:17:50.351151 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.351054 2563 kubelet.go:398] "Adding apiserver pod source" Jul 21 06:17:50.351151 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.351062 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 06:17:50.351396 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.351384 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 06:17:50.351428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.351410 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 06:17:50.353248 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.353231 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 06:17:50.354722 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.354708 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 06:17:50.354778 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.354738 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 06:17:50.356063 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356051 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 06:17:50.356063 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356066 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 06:17:50.356159 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356072 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 06:17:50.356159 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356078 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 06:17:50.356159 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356087 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 06:17:50.356159 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356093 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 06:17:50.356159 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356103 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 06:17:50.356159 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356111 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 06:17:50.356159 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356118 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 06:17:50.356159 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356123 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 06:17:50.356159 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356131 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 06:17:50.356159 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356149 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 06:17:50.356858 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356845 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 06:17:50.356858 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.356858 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 06:17:50.359578 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.359546 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 06:17:50.359578 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.359557 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-159.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 06:17:50.359578 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.359567 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-159.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 06:17:50.360381 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.360366 2563 server.go:1268] "Started kubelet" Jul 21 06:17:50.360565 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.360514 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 06:17:50.360656 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.360586 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 06:17:50.361248 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.361202 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 06:17:50.361430 ip-10-0-140-159 systemd[1]: Started Kubernetes Kubelet. Jul 21 06:17:50.362285 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.362203 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 06:17:50.362867 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.362854 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 21 06:17:50.365778 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.365752 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 06:17:50.366687 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.366671 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 06:17:50.366740 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.366681 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 06:17:50.367021 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.366998 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:50.367195 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.367176 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 06:17:50.367195 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.367196 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 06:17:50.367345 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.367200 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rc7cr" Jul 21 06:17:50.367345 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.367257 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 06:17:50.367512 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.367490 2563 factory.go:55] Registering systemd factory Jul 21 06:17:50.367573 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.367519 2563 factory.go:223] Registration of the systemd container factory successfully Jul 21 06:17:50.367815 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.367800 2563 factory.go:153] Registering CRI-O factory Jul 21 06:17:50.367901 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.367818 2563 factory.go:223] Registration of the crio container factory successfully Jul 21 06:17:50.367901 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.367854 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 21 06:17:50.367901 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.367861 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 21 06:17:50.367901 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.367868 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 06:17:50.367901 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.367889 2563 factory.go:103] Registering Raw factory Jul 21 06:17:50.367901 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.367901 2563 manager.go:1196] Started watching for new ooms in manager Jul 21 06:17:50.368379 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.368368 2563 manager.go:349] Starting recovery of all containers Jul 21 06:17:50.374073 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.374041 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 21 06:17:50.374390 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.374367 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rc7cr" Jul 21 06:17:50.378233 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.378182 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-159.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 21 06:17:50.379526 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.378186 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-159.ec2.internal.18c439965fc76e4a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-159.ec2.internal,UID:ip-10-0-140-159.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-159.ec2.internal,},FirstTimestamp:2026-07-21 06:17:50.36034209 +0000 UTC m=+0.305980739,LastTimestamp:2026-07-21 06:17:50.36034209 +0000 UTC m=+0.305980739,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-159.ec2.internal,}" Jul 21 06:17:50.379971 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.379954 2563 manager.go:354] Recovery completed Jul 21 06:17:50.384593 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.384580 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 06:17:50.386383 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.386366 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientMemory" Jul 21 06:17:50.386447 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.386393 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 06:17:50.386447 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.386403 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientPID" Jul 21 06:17:50.386854 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.386840 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 06:17:50.386854 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.386853 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 06:17:50.386968 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.386866 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 21 06:17:50.386968 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.386912 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 06:17:50.387613 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.387599 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientMemory" Jul 21 06:17:50.387687 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.387629 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 06:17:50.387687 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.387632 2563 policy_none.go:49] "None policy: Start" Jul 21 06:17:50.387687 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.387639 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientPID" Jul 21 06:17:50.387687 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.387644 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 06:17:50.387687 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.387654 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 06:17:50.388464 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.388452 2563 policy_none.go:47] "Start" Jul 21 06:17:50.420034 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.420020 2563 manager.go:337] "Starting Device Plugin manager" Jul 21 06:17:50.420105 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.420055 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 06:17:50.420105 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.420068 2563 server.go:87] "Starting device plugin registration server" Jul 21 06:17:50.420329 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.420313 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 06:17:50.420407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.420330 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 06:17:50.420456 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.420414 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 06:17:50.420992 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.420911 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 06:17:50.420992 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.420935 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 06:17:50.420992 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.420946 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 06:17:50.420992 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.420965 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:50.478557 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.478527 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 06:17:50.479887 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.479875 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 06:17:50.479942 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.479913 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 06:17:50.479942 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.479940 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 06:17:50.480038 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.479976 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 06:17:50.482436 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.482422 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 06:17:50.521335 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.521310 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 06:17:50.522264 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.522249 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientMemory" Jul 21 06:17:50.522324 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.522278 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 06:17:50.522324 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.522288 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientPID" Jul 21 06:17:50.522324 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.522309 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.527937 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.527923 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.527980 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.527944 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-159.ec2.internal\": node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:50.538608 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.538589 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:50.580703 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.580647 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-159.ec2.internal"] Jul 21 06:17:50.580778 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.580706 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 06:17:50.582133 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.582117 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientMemory" Jul 21 06:17:50.582188 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.582146 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 06:17:50.582188 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.582156 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientPID" Jul 21 06:17:50.583187 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.583173 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 06:17:50.583320 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.583306 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.583380 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.583344 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 06:17:50.583871 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.583858 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientMemory" Jul 21 06:17:50.583951 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.583890 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 06:17:50.583951 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.583904 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientPID" Jul 21 06:17:50.583951 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.583861 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientMemory" Jul 21 06:17:50.584088 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.583962 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 06:17:50.584088 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.583975 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientPID" Jul 21 06:17:50.584994 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.584977 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.585043 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.585006 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 06:17:50.585648 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.585631 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientMemory" Jul 21 06:17:50.585733 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.585660 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 06:17:50.585733 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.585672 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeHasSufficientPID" Jul 21 06:17:50.602764 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.602746 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-159.ec2.internal\" not found" node="ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.606676 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.606661 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-159.ec2.internal\" not found" node="ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.639225 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.639171 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:50.669927 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.669891 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e682834e4c02794e4aba5128ec0c7ad6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal\" (UID: \"e682834e4c02794e4aba5128ec0c7ad6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.670035 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.669930 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e682834e4c02794e4aba5128ec0c7ad6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal\" (UID: \"e682834e4c02794e4aba5128ec0c7ad6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.670035 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.669954 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0dad6f1e84139608ab928a8fbb218961-config\") pod \"kube-apiserver-proxy-ip-10-0-140-159.ec2.internal\" (UID: \"0dad6f1e84139608ab928a8fbb218961\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.739859 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.739839 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:50.770281 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.770262 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0dad6f1e84139608ab928a8fbb218961-config\") pod \"kube-apiserver-proxy-ip-10-0-140-159.ec2.internal\" (UID: \"0dad6f1e84139608ab928a8fbb218961\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.770363 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.770285 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e682834e4c02794e4aba5128ec0c7ad6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal\" (UID: \"e682834e4c02794e4aba5128ec0c7ad6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.770363 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.770301 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e682834e4c02794e4aba5128ec0c7ad6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal\" (UID: \"e682834e4c02794e4aba5128ec0c7ad6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.770363 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.770325 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e682834e4c02794e4aba5128ec0c7ad6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal\" (UID: \"e682834e4c02794e4aba5128ec0c7ad6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.770476 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.770362 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e682834e4c02794e4aba5128ec0c7ad6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal\" (UID: \"e682834e4c02794e4aba5128ec0c7ad6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.770476 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.770366 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0dad6f1e84139608ab928a8fbb218961-config\") pod \"kube-apiserver-proxy-ip-10-0-140-159.ec2.internal\" (UID: \"0dad6f1e84139608ab928a8fbb218961\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.840575 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.840527 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:50.905133 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.905095 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.909771 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:50.909758 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-159.ec2.internal" Jul 21 06:17:50.941169 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:50.941149 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:51.041670 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:51.041642 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:51.142158 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:51.142098 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:51.172204 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.172180 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 06:17:51.242275 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:51.242246 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:51.315770 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.315749 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 06:17:51.316293 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.315883 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 06:17:51.316293 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.315940 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 06:17:51.343042 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:51.343011 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:51.377712 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.377680 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 06:12:51 +0000 UTC" deadline="2027-12-20 18:44:28.309811109 +0000 UTC" Jul 21 06:17:51.377712 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.377707 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12420h26m36.932106828s" Jul 21 06:17:51.401465 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:51.401402 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode682834e4c02794e4aba5128ec0c7ad6.slice/crio-6059bd8151bafbc5406b6b91b34a0e7318f45ac5d3559d243b56aac94f395004 WatchSource:0}: Error finding container 6059bd8151bafbc5406b6b91b34a0e7318f45ac5d3559d243b56aac94f395004: Status 404 returned error can't find the container with id 6059bd8151bafbc5406b6b91b34a0e7318f45ac5d3559d243b56aac94f395004 Jul 21 06:17:51.401830 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:51.401810 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dad6f1e84139608ab928a8fbb218961.slice/crio-bcd3f4250fe59c132e76b3194533e80808c5dcdb3041f9568d145c843ba16290 WatchSource:0}: Error finding container bcd3f4250fe59c132e76b3194533e80808c5dcdb3041f9568d145c843ba16290: Status 404 returned error can't find the container with id bcd3f4250fe59c132e76b3194533e80808c5dcdb3041f9568d145c843ba16290 Jul 21 06:17:51.406624 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.406608 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 06:17:51.443429 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:51.443409 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:51.483966 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.483923 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-159.ec2.internal" event={"ID":"0dad6f1e84139608ab928a8fbb218961","Type":"ContainerStarted","Data":"bcd3f4250fe59c132e76b3194533e80808c5dcdb3041f9568d145c843ba16290"} Jul 21 06:17:51.485411 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.485382 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal" event={"ID":"e682834e4c02794e4aba5128ec0c7ad6","Type":"ContainerStarted","Data":"6059bd8151bafbc5406b6b91b34a0e7318f45ac5d3559d243b56aac94f395004"} Jul 21 06:17:51.543531 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:51.543509 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:51.644000 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:51.643969 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:51.681503 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.681435 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 06:17:51.749029 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:51.748996 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-159.ec2.internal\" not found" Jul 21 06:17:51.752174 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.752156 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 06:17:51.766936 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.766912 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal" Jul 21 06:17:51.777843 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.777823 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 06:17:51.778896 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.778882 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-159.ec2.internal" Jul 21 06:17:51.787056 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:51.787038 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 06:17:52.352706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.352672 2563 apiserver.go:52] "Watching apiserver" Jul 21 06:17:52.362632 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.362604 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 06:17:52.363194 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.363167 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-ztv8x","openshift-multus/multus-additional-cni-plugins-gzpql","openshift-multus/network-metrics-daemon-l2867","openshift-network-diagnostics/network-check-target-5sldl","openshift-network-operator/iptables-alerter-njwwp","openshift-ovn-kubernetes/ovnkube-node-8qbzm","kube-system/global-pull-secret-syncer-kwzdp","kube-system/konnectivity-agent-nzgkk","kube-system/kube-apiserver-proxy-ip-10-0-140-159.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf","openshift-dns/node-resolver-ks8rh","openshift-image-registry/node-ca-6dtks","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal","openshift-multus/multus-fkwwc"] Jul 21 06:17:52.364986 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.364623 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:52.364986 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.364708 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:17:52.366011 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.365985 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ks8rh" Jul 21 06:17:52.366905 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.366878 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:17:52.367015 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.366959 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:17:52.367963 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.367944 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:17:52.368116 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.368092 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:17:52.368191 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.368153 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 06:17:52.371152 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.369244 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-khhfb\"" Jul 21 06:17:52.371152 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.369260 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 06:17:52.371152 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.369318 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 06:17:52.371152 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.370034 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-njwwp" Jul 21 06:17:52.371432 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.371268 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.372596 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.372568 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.373121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.373097 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 06:17:52.373261 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.373097 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 06:17:52.373450 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.373434 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 06:17:52.373704 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.373681 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4b9c6\"" Jul 21 06:17:52.374222 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.374000 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nzgkk" Jul 21 06:17:52.374222 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.374072 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 06:17:52.374626 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.374516 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 06:17:52.375412 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.375384 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 06:17:52.375660 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.375642 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.377512 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.376362 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 06:17:52.377512 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.376727 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dwspp\"" Jul 21 06:17:52.377512 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.377187 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 06:17:52.377876 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.377855 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 06:17:52.378029 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.378011 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 06:17:52.378130 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.378107 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 06:17:52.378182 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.378161 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 06:17:52.378439 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.378410 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 06:17:52.378576 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.378514 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c2w5z\"" Jul 21 06:17:52.378576 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.378524 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 06:17:52.378695 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.378670 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9jq8w\"" Jul 21 06:17:52.379478 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.379458 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tmdn\" (UniqueName: \"kubernetes.io/projected/42908595-6e16-4688-b583-504934328eb7-kube-api-access-6tmdn\") pod \"network-metrics-daemon-l2867\" (UID: \"42908595-6e16-4688-b583-504934328eb7\") " pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:17:52.379573 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.379500 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzgpj\" (UniqueName: \"kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj\") pod \"network-check-target-5sldl\" (UID: \"a69c086c-f31f-416b-a386-0a3e3e458acb\") " pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:17:52.379573 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.379533 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkdcj\" (UniqueName: \"kubernetes.io/projected/934f9315-0fc6-4f88-88fe-7b9b5cd6335d-kube-api-access-tkdcj\") pod \"iptables-alerter-njwwp\" (UID: \"934f9315-0fc6-4f88-88fe-7b9b5cd6335d\") " pod="openshift-network-operator/iptables-alerter-njwwp" Jul 21 06:17:52.379573 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.379561 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs\") pod \"network-metrics-daemon-l2867\" (UID: \"42908595-6e16-4688-b583-504934328eb7\") " pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:17:52.379786 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.379593 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/934f9315-0fc6-4f88-88fe-7b9b5cd6335d-iptables-alerter-script\") pod \"iptables-alerter-njwwp\" (UID: \"934f9315-0fc6-4f88-88fe-7b9b5cd6335d\") " pod="openshift-network-operator/iptables-alerter-njwwp" Jul 21 06:17:52.379786 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.379627 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/934f9315-0fc6-4f88-88fe-7b9b5cd6335d-host-slash\") pod \"iptables-alerter-njwwp\" (UID: \"934f9315-0fc6-4f88-88fe-7b9b5cd6335d\") " pod="openshift-network-operator/iptables-alerter-njwwp" Jul 21 06:17:52.379786 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.379661 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ee4a567-fe9e-4346-ac51-1d78dede57d1-kubelet-config\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:52.379786 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.379693 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ee4a567-fe9e-4346-ac51-1d78dede57d1-dbus\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:52.379786 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.379721 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:52.379786 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.379756 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/512070fa-797a-4c81-92f3-0510a4c7cfbb-hosts-file\") pod \"node-resolver-ks8rh\" (UID: \"512070fa-797a-4c81-92f3-0510a4c7cfbb\") " pod="openshift-dns/node-resolver-ks8rh" Jul 21 06:17:52.380077 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.379815 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/512070fa-797a-4c81-92f3-0510a4c7cfbb-tmp-dir\") pod \"node-resolver-ks8rh\" (UID: \"512070fa-797a-4c81-92f3-0510a4c7cfbb\") " pod="openshift-dns/node-resolver-ks8rh" Jul 21 06:17:52.380077 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.379964 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r28z\" (UniqueName: \"kubernetes.io/projected/512070fa-797a-4c81-92f3-0510a4c7cfbb-kube-api-access-6r28z\") pod \"node-resolver-ks8rh\" (UID: \"512070fa-797a-4c81-92f3-0510a4c7cfbb\") " pod="openshift-dns/node-resolver-ks8rh" Jul 21 06:17:52.380077 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.380059 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 06:17:52.380366 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.380347 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.380590 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.380570 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 06:17:52.381843 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.381822 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m87r8\"" Jul 21 06:17:52.382708 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.382683 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 06:17:52.383568 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.383545 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2zdfx\"" Jul 21 06:17:52.384037 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.384017 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 06:17:52.384243 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.384225 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 06:17:52.384349 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.384334 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6dtks" Jul 21 06:17:52.384561 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.384534 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 06:17:52.384639 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.384627 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.385730 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.385711 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 06:17:52.389259 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.387031 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 06:17:52.389259 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.387154 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 06:17:52.389259 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.387312 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 06:17:52.389259 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.387656 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 06:17:52.389259 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.387675 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z24dr\"" Jul 21 06:17:52.389259 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.387674 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 06:17:52.389259 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.387713 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p457f\"" Jul 21 06:17:52.402728 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.402707 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dlfcv" Jul 21 06:17:52.413439 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.413421 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dlfcv" Jul 21 06:17:52.468023 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.468001 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 06:17:52.481168 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481134 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzgpj\" (UniqueName: \"kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj\") pod \"network-check-target-5sldl\" (UID: \"a69c086c-f31f-416b-a386-0a3e3e458acb\") " pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:17:52.481310 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481180 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-etc-openvswitch\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.481310 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481203 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-run-openvswitch\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.481310 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481247 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/add515ac-8a6a-4e1a-a0ad-b555677f3907-ovnkube-script-lib\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.481310 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481274 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cf0536e7-62ba-4202-99af-2d8c13d615f2-os-release\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.481310 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481302 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-run-netns\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.481550 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481363 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-var-lib-cni-multus\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.481550 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481389 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf0536e7-62ba-4202-99af-2d8c13d615f2-system-cni-dir\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.481550 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481441 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r5km\" (UniqueName: \"kubernetes.io/projected/cf0536e7-62ba-4202-99af-2d8c13d615f2-kube-api-access-8r5km\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.481550 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481483 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-sysconfig\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.481550 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481499 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-sysctl-conf\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.481550 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481517 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-systemd\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.481795 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481556 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-multus-conf-dir\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.481795 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481580 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/88a16c6f-e274-49dd-952e-065d7ba5b89c-multus-daemon-config\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.481795 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481610 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-socket-dir\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.481795 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481646 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-run-systemd\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.481795 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481696 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvfhd\" (UniqueName: \"kubernetes.io/projected/add515ac-8a6a-4e1a-a0ad-b555677f3907-kube-api-access-lvfhd\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.481795 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481733 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-modprobe-d\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.481795 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481760 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-var-lib-kubelet\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.481795 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481786 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-run-multus-certs\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.482121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481812 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-etc-selinux\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.482121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481837 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs9hb\" (UniqueName: \"kubernetes.io/projected/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-kube-api-access-qs9hb\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.482121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481863 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/add515ac-8a6a-4e1a-a0ad-b555677f3907-env-overrides\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.482121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481886 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b9eedde-6415-425e-9223-d74b2a3f74cb-host\") pod \"node-ca-6dtks\" (UID: \"7b9eedde-6415-425e-9223-d74b2a3f74cb\") " pod="openshift-image-registry/node-ca-6dtks" Jul 21 06:17:52.482121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481914 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b9eedde-6415-425e-9223-d74b2a3f74cb-serviceca\") pod \"node-ca-6dtks\" (UID: \"7b9eedde-6415-425e-9223-d74b2a3f74cb\") " pod="openshift-image-registry/node-ca-6dtks" Jul 21 06:17:52.482121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481945 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tmdn\" (UniqueName: \"kubernetes.io/projected/42908595-6e16-4688-b583-504934328eb7-kube-api-access-6tmdn\") pod \"network-metrics-daemon-l2867\" (UID: \"42908595-6e16-4688-b583-504934328eb7\") " pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:17:52.482121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.481972 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf0536e7-62ba-4202-99af-2d8c13d615f2-cnibin\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.482121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482000 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/934f9315-0fc6-4f88-88fe-7b9b5cd6335d-iptables-alerter-script\") pod \"iptables-alerter-njwwp\" (UID: \"934f9315-0fc6-4f88-88fe-7b9b5cd6335d\") " pod="openshift-network-operator/iptables-alerter-njwwp" Jul 21 06:17:52.482121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482025 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/934f9315-0fc6-4f88-88fe-7b9b5cd6335d-host-slash\") pod \"iptables-alerter-njwwp\" (UID: \"934f9315-0fc6-4f88-88fe-7b9b5cd6335d\") " pod="openshift-network-operator/iptables-alerter-njwwp" Jul 21 06:17:52.482121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482051 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-kubernetes\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.482121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482079 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-hostroot\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.482121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482101 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dd4adafa-b828-4ddc-8668-036cb8960d77-agent-certs\") pod \"konnectivity-agent-nzgkk\" (UID: \"dd4adafa-b828-4ddc-8668-036cb8960d77\") " pod="kube-system/konnectivity-agent-nzgkk" Jul 21 06:17:52.482639 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482136 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/512070fa-797a-4c81-92f3-0510a4c7cfbb-hosts-file\") pod \"node-resolver-ks8rh\" (UID: \"512070fa-797a-4c81-92f3-0510a4c7cfbb\") " pod="openshift-dns/node-resolver-ks8rh" Jul 21 06:17:52.482639 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482160 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/512070fa-797a-4c81-92f3-0510a4c7cfbb-tmp-dir\") pod \"node-resolver-ks8rh\" (UID: \"512070fa-797a-4c81-92f3-0510a4c7cfbb\") " pod="openshift-dns/node-resolver-ks8rh" Jul 21 06:17:52.482639 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482184 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-run-netns\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.482639 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482207 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf0536e7-62ba-4202-99af-2d8c13d615f2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.482639 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482229 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-cnibin\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.482639 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482271 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-run-k8s-cni-cncf-io\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.482639 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482359 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/934f9315-0fc6-4f88-88fe-7b9b5cd6335d-host-slash\") pod \"iptables-alerter-njwwp\" (UID: \"934f9315-0fc6-4f88-88fe-7b9b5cd6335d\") " pod="openshift-network-operator/iptables-alerter-njwwp" Jul 21 06:17:52.482639 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482415 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/512070fa-797a-4c81-92f3-0510a4c7cfbb-hosts-file\") pod \"node-resolver-ks8rh\" (UID: \"512070fa-797a-4c81-92f3-0510a4c7cfbb\") " pod="openshift-dns/node-resolver-ks8rh" Jul 21 06:17:52.482946 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482718 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/512070fa-797a-4c81-92f3-0510a4c7cfbb-tmp-dir\") pod \"node-resolver-ks8rh\" (UID: \"512070fa-797a-4c81-92f3-0510a4c7cfbb\") " pod="openshift-dns/node-resolver-ks8rh" Jul 21 06:17:52.482946 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482810 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-slash\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.483043 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.482999 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.483043 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483024 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/934f9315-0fc6-4f88-88fe-7b9b5cd6335d-iptables-alerter-script\") pod \"iptables-alerter-njwwp\" (UID: \"934f9315-0fc6-4f88-88fe-7b9b5cd6335d\") " pod="openshift-network-operator/iptables-alerter-njwwp" Jul 21 06:17:52.483171 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483045 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf0536e7-62ba-4202-99af-2d8c13d615f2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.483171 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483102 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkdcj\" (UniqueName: \"kubernetes.io/projected/934f9315-0fc6-4f88-88fe-7b9b5cd6335d-kube-api-access-tkdcj\") pod \"iptables-alerter-njwwp\" (UID: \"934f9315-0fc6-4f88-88fe-7b9b5cd6335d\") " pod="openshift-network-operator/iptables-alerter-njwwp" Jul 21 06:17:52.483273 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483167 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-run-ovn-kubernetes\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.483338 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483269 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/add515ac-8a6a-4e1a-a0ad-b555677f3907-ovnkube-config\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.483338 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483313 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-os-release\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.483412 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483344 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88a16c6f-e274-49dd-952e-065d7ba5b89c-cni-binary-copy\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.483412 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483364 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-multus-socket-dir-parent\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.483412 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483387 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ee4a567-fe9e-4346-ac51-1d78dede57d1-kubelet-config\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:52.483412 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483405 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:52.483551 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483433 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-var-lib-openvswitch\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.483551 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483451 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-run-ovn\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.483551 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483489 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf0536e7-62ba-4202-99af-2d8c13d615f2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.484304 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.483504 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-sys\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.484415 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.484296 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.484415 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.484340 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 06:17:52.484415 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.484372 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ee4a567-fe9e-4346-ac51-1d78dede57d1-kubelet-config\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:52.484579 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.484426 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5763dac6-5176-4058-ae43-782e71cc676d-tmp\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.484579 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.484491 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret podName:2ee4a567-fe9e-4346-ac51-1d78dede57d1 nodeName:}" failed. No retries permitted until 2026-07-21 06:17:52.984458677 +0000 UTC m=+2.930097337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret") pod "global-pull-secret-syncer-kwzdp" (UID: "2ee4a567-fe9e-4346-ac51-1d78dede57d1") : object "kube-system"/"original-pull-secret" not registered Jul 21 06:17:52.484579 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.484516 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dd4adafa-b828-4ddc-8668-036cb8960d77-konnectivity-ca\") pod \"konnectivity-agent-nzgkk\" (UID: \"dd4adafa-b828-4ddc-8668-036cb8960d77\") " pod="kube-system/konnectivity-agent-nzgkk" Jul 21 06:17:52.484717 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.484588 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-registration-dir\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.484717 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.484632 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpf55\" (UniqueName: \"kubernetes.io/projected/7b9eedde-6415-425e-9223-d74b2a3f74cb-kube-api-access-dpf55\") pod \"node-ca-6dtks\" (UID: \"7b9eedde-6415-425e-9223-d74b2a3f74cb\") " pod="openshift-image-registry/node-ca-6dtks" Jul 21 06:17:52.484717 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.484678 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r28z\" (UniqueName: \"kubernetes.io/projected/512070fa-797a-4c81-92f3-0510a4c7cfbb-kube-api-access-6r28z\") pod \"node-resolver-ks8rh\" (UID: \"512070fa-797a-4c81-92f3-0510a4c7cfbb\") " pod="openshift-dns/node-resolver-ks8rh" Jul 21 06:17:52.484847 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.484738 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-kubelet\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.484903 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.484854 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-log-socket\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.484959 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.484909 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf0536e7-62ba-4202-99af-2d8c13d615f2-cni-binary-copy\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.484959 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.484948 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-host\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.485067 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.484984 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-sys-fs\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.485067 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485020 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-node-log\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.485067 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485055 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-cni-bin\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.485188 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485084 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/add515ac-8a6a-4e1a-a0ad-b555677f3907-ovn-node-metrics-cert\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.485188 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485130 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-sysctl-d\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.485324 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485185 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmz64\" (UniqueName: \"kubernetes.io/projected/5763dac6-5176-4058-ae43-782e71cc676d-kube-api-access-mmz64\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.485324 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485266 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-var-lib-cni-bin\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.485324 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485313 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs\") pod \"network-metrics-daemon-l2867\" (UID: \"42908595-6e16-4688-b583-504934328eb7\") " pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:17:52.485480 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485342 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ee4a567-fe9e-4346-ac51-1d78dede57d1-dbus\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:52.485480 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485380 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-systemd-units\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.485480 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485415 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-run\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.485480 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.485442 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:17:52.485480 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485448 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-lib-modules\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.485740 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485485 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-system-cni-dir\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.485740 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.485510 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs podName:42908595-6e16-4688-b583-504934328eb7 nodeName:}" failed. No retries permitted until 2026-07-21 06:17:52.98548392 +0000 UTC m=+2.931122571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs") pod "network-metrics-daemon-l2867" (UID: "42908595-6e16-4688-b583-504934328eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:17:52.485740 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485562 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-etc-kubernetes\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.485740 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485596 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-cni-netd\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.485740 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485621 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-var-lib-kubelet\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.485740 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485653 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5763dac6-5176-4058-ae43-782e71cc676d-etc-tuned\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.485740 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485662 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ee4a567-fe9e-4346-ac51-1d78dede57d1-dbus\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:52.485740 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485683 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-multus-cni-dir\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.485740 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485740 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swk76\" (UniqueName: \"kubernetes.io/projected/88a16c6f-e274-49dd-952e-065d7ba5b89c-kube-api-access-swk76\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.486176 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.485805 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-device-dir\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.488072 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.488054 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 06:17:52.488072 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.488076 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 06:17:52.488195 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.488087 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzgpj for pod openshift-network-diagnostics/network-check-target-5sldl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:17:52.488195 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.488137 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj podName:a69c086c-f31f-416b-a386-0a3e3e458acb nodeName:}" failed. No retries permitted until 2026-07-21 06:17:52.988125005 +0000 UTC m=+2.933763641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tzgpj" (UniqueName: "kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj") pod "network-check-target-5sldl" (UID: "a69c086c-f31f-416b-a386-0a3e3e458acb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:17:52.488923 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.488900 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 06:17:52.493257 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.493231 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkdcj\" (UniqueName: \"kubernetes.io/projected/934f9315-0fc6-4f88-88fe-7b9b5cd6335d-kube-api-access-tkdcj\") pod \"iptables-alerter-njwwp\" (UID: \"934f9315-0fc6-4f88-88fe-7b9b5cd6335d\") " pod="openshift-network-operator/iptables-alerter-njwwp" Jul 21 06:17:52.493356 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.493273 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tmdn\" (UniqueName: \"kubernetes.io/projected/42908595-6e16-4688-b583-504934328eb7-kube-api-access-6tmdn\") pod \"network-metrics-daemon-l2867\" (UID: \"42908595-6e16-4688-b583-504934328eb7\") " pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:17:52.493463 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.493442 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r28z\" (UniqueName: \"kubernetes.io/projected/512070fa-797a-4c81-92f3-0510a4c7cfbb-kube-api-access-6r28z\") pod \"node-resolver-ks8rh\" (UID: \"512070fa-797a-4c81-92f3-0510a4c7cfbb\") " pod="openshift-dns/node-resolver-ks8rh" Jul 21 06:17:52.521116 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.521091 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 06:17:52.586462 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586426 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-sysconfig\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.586462 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586461 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-sysctl-conf\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.586662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-systemd\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.586662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586505 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-multus-conf-dir\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.586662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586532 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/88a16c6f-e274-49dd-952e-065d7ba5b89c-multus-daemon-config\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.586662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586562 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-sysconfig\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.586662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586569 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-socket-dir\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.586662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586593 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-run-systemd\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.586662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586617 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-multus-conf-dir\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.586662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586620 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvfhd\" (UniqueName: \"kubernetes.io/projected/add515ac-8a6a-4e1a-a0ad-b555677f3907-kube-api-access-lvfhd\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.586662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586654 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-modprobe-d\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.586948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586680 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-var-lib-kubelet\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.586948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586704 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-run-multus-certs\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.586948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586730 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-etc-selinux\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.586948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586754 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs9hb\" (UniqueName: \"kubernetes.io/projected/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-kube-api-access-qs9hb\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.586948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586777 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/add515ac-8a6a-4e1a-a0ad-b555677f3907-env-overrides\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.586948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586801 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b9eedde-6415-425e-9223-d74b2a3f74cb-host\") pod \"node-ca-6dtks\" (UID: \"7b9eedde-6415-425e-9223-d74b2a3f74cb\") " pod="openshift-image-registry/node-ca-6dtks" Jul 21 06:17:52.586948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586825 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b9eedde-6415-425e-9223-d74b2a3f74cb-serviceca\") pod \"node-ca-6dtks\" (UID: \"7b9eedde-6415-425e-9223-d74b2a3f74cb\") " pod="openshift-image-registry/node-ca-6dtks" Jul 21 06:17:52.586948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586853 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf0536e7-62ba-4202-99af-2d8c13d615f2-cnibin\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.586948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586881 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-kubernetes\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.586948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586902 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-hostroot\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.586948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586926 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dd4adafa-b828-4ddc-8668-036cb8960d77-agent-certs\") pod \"konnectivity-agent-nzgkk\" (UID: \"dd4adafa-b828-4ddc-8668-036cb8960d77\") " pod="kube-system/konnectivity-agent-nzgkk" Jul 21 06:17:52.587270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586951 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-run-netns\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.587270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586974 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf0536e7-62ba-4202-99af-2d8c13d615f2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.587270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.586999 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-cnibin\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.587270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587024 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-run-k8s-cni-cncf-io\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.587270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587050 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-slash\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.587270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587076 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.587270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587105 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf0536e7-62ba-4202-99af-2d8c13d615f2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.587270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587135 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-run-ovn-kubernetes\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.587270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587159 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/add515ac-8a6a-4e1a-a0ad-b555677f3907-ovnkube-config\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.587270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587187 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-os-release\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.587270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587210 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88a16c6f-e274-49dd-952e-065d7ba5b89c-cni-binary-copy\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.587270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587237 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-multus-socket-dir-parent\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587311 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-var-lib-openvswitch\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587344 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-run-ovn\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587371 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf0536e7-62ba-4202-99af-2d8c13d615f2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587381 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-hostroot\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587396 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-sys\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587386 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587427 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587487 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/88a16c6f-e274-49dd-952e-065d7ba5b89c-multus-daemon-config\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587498 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-socket-dir\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587517 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-cnibin\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587543 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-run-systemd\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587540 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf0536e7-62ba-4202-99af-2d8c13d615f2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587558 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-sys\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587562 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-run-k8s-cni-cncf-io\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587543 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-os-release\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587605 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-systemd\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587605 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-run-ovn-kubernetes\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.587911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587611 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-run-netns\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587651 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf0536e7-62ba-4202-99af-2d8c13d615f2-cnibin\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587666 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-var-lib-openvswitch\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587718 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-multus-socket-dir-parent\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587760 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-var-lib-kubelet\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587796 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5763dac6-5176-4058-ae43-782e71cc676d-tmp\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587858 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dd4adafa-b828-4ddc-8668-036cb8960d77-konnectivity-ca\") pod \"konnectivity-agent-nzgkk\" (UID: \"dd4adafa-b828-4ddc-8668-036cb8960d77\") " pod="kube-system/konnectivity-agent-nzgkk" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587867 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-etc-selinux\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587887 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-registration-dir\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587908 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b9eedde-6415-425e-9223-d74b2a3f74cb-host\") pod \"node-ca-6dtks\" (UID: \"7b9eedde-6415-425e-9223-d74b2a3f74cb\") " pod="openshift-image-registry/node-ca-6dtks" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587915 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpf55\" (UniqueName: \"kubernetes.io/projected/7b9eedde-6415-425e-9223-d74b2a3f74cb-kube-api-access-dpf55\") pod \"node-ca-6dtks\" (UID: \"7b9eedde-6415-425e-9223-d74b2a3f74cb\") " pod="openshift-image-registry/node-ca-6dtks" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587945 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-kubelet\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587968 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-log-socket\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587997 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf0536e7-62ba-4202-99af-2d8c13d615f2-cni-binary-copy\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.587997 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-modprobe-d\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588029 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-slash\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588035 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf0536e7-62ba-4202-99af-2d8c13d615f2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.588850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588067 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-kubelet\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588075 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-run-multus-certs\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588111 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-host\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588139 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-sys-fs\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588149 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b9eedde-6415-425e-9223-d74b2a3f74cb-serviceca\") pod \"node-ca-6dtks\" (UID: \"7b9eedde-6415-425e-9223-d74b2a3f74cb\") " pod="openshift-image-registry/node-ca-6dtks" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588168 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-node-log\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588153 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88a16c6f-e274-49dd-952e-065d7ba5b89c-cni-binary-copy\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588177 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/add515ac-8a6a-4e1a-a0ad-b555677f3907-ovnkube-config\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588176 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-sysctl-conf\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588110 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-log-socket\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588201 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-kubernetes\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588230 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-node-log\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588234 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-cni-bin\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588240 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-sys-fs\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588277 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-run-ovn\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588311 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-registration-dir\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588327 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588328 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/add515ac-8a6a-4e1a-a0ad-b555677f3907-ovn-node-metrics-cert\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.589706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588361 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-cni-bin\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588387 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-sysctl-d\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588402 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-host\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588415 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmz64\" (UniqueName: \"kubernetes.io/projected/5763dac6-5176-4058-ae43-782e71cc676d-kube-api-access-mmz64\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588453 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-var-lib-cni-bin\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588502 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-var-lib-cni-bin\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588539 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-etc-sysctl-d\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588540 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-systemd-units\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588572 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-systemd-units\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588572 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/add515ac-8a6a-4e1a-a0ad-b555677f3907-env-overrides\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588615 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-run\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588640 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-lib-modules\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588646 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf0536e7-62ba-4202-99af-2d8c13d615f2-cni-binary-copy\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588677 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-system-cni-dir\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588709 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-etc-kubernetes\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588735 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-cni-netd\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588763 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-var-lib-kubelet\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588783 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-system-cni-dir\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.590610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588789 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5763dac6-5176-4058-ae43-782e71cc676d-etc-tuned\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588793 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-run\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588848 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-multus-cni-dir\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588875 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swk76\" (UniqueName: \"kubernetes.io/projected/88a16c6f-e274-49dd-952e-065d7ba5b89c-kube-api-access-swk76\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588878 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-host-cni-netd\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588895 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-lib-modules\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588908 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5763dac6-5176-4058-ae43-782e71cc676d-var-lib-kubelet\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588912 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-device-dir\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588936 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-etc-kubernetes\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.588942 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-multus-cni-dir\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589030 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-device-dir\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589038 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf0536e7-62ba-4202-99af-2d8c13d615f2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589110 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-etc-openvswitch\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589145 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-etc-openvswitch\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589157 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-run-openvswitch\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589187 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/add515ac-8a6a-4e1a-a0ad-b555677f3907-run-openvswitch\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589199 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/add515ac-8a6a-4e1a-a0ad-b555677f3907-ovnkube-script-lib\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589227 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cf0536e7-62ba-4202-99af-2d8c13d615f2-os-release\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.591505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589253 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-run-netns\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.592155 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589279 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-var-lib-cni-multus\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.592155 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589355 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-var-lib-cni-multus\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.592155 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589375 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cf0536e7-62ba-4202-99af-2d8c13d615f2-os-release\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.592155 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589401 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88a16c6f-e274-49dd-952e-065d7ba5b89c-host-run-netns\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.592155 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589408 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf0536e7-62ba-4202-99af-2d8c13d615f2-system-cni-dir\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.592155 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589437 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r5km\" (UniqueName: \"kubernetes.io/projected/cf0536e7-62ba-4202-99af-2d8c13d615f2-kube-api-access-8r5km\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.592155 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589525 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dd4adafa-b828-4ddc-8668-036cb8960d77-konnectivity-ca\") pod \"konnectivity-agent-nzgkk\" (UID: \"dd4adafa-b828-4ddc-8668-036cb8960d77\") " pod="kube-system/konnectivity-agent-nzgkk" Jul 21 06:17:52.592155 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589591 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf0536e7-62ba-4202-99af-2d8c13d615f2-system-cni-dir\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.592155 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.589912 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/add515ac-8a6a-4e1a-a0ad-b555677f3907-ovnkube-script-lib\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.592155 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.591041 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dd4adafa-b828-4ddc-8668-036cb8960d77-agent-certs\") pod \"konnectivity-agent-nzgkk\" (UID: \"dd4adafa-b828-4ddc-8668-036cb8960d77\") " pod="kube-system/konnectivity-agent-nzgkk" Jul 21 06:17:52.592155 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.591747 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5763dac6-5176-4058-ae43-782e71cc676d-etc-tuned\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.592155 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.592052 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/add515ac-8a6a-4e1a-a0ad-b555677f3907-ovn-node-metrics-cert\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.592591 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.592452 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5763dac6-5176-4058-ae43-782e71cc676d-tmp\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.597691 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.597665 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvfhd\" (UniqueName: \"kubernetes.io/projected/add515ac-8a6a-4e1a-a0ad-b555677f3907-kube-api-access-lvfhd\") pod \"ovnkube-node-8qbzm\" (UID: \"add515ac-8a6a-4e1a-a0ad-b555677f3907\") " pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.598559 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.598532 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swk76\" (UniqueName: \"kubernetes.io/projected/88a16c6f-e274-49dd-952e-065d7ba5b89c-kube-api-access-swk76\") pod \"multus-fkwwc\" (UID: \"88a16c6f-e274-49dd-952e-065d7ba5b89c\") " pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.599003 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.598904 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r5km\" (UniqueName: \"kubernetes.io/projected/cf0536e7-62ba-4202-99af-2d8c13d615f2-kube-api-access-8r5km\") pod \"multus-additional-cni-plugins-gzpql\" (UID: \"cf0536e7-62ba-4202-99af-2d8c13d615f2\") " pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.599103 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.599017 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpf55\" (UniqueName: \"kubernetes.io/projected/7b9eedde-6415-425e-9223-d74b2a3f74cb-kube-api-access-dpf55\") pod \"node-ca-6dtks\" (UID: \"7b9eedde-6415-425e-9223-d74b2a3f74cb\") " pod="openshift-image-registry/node-ca-6dtks" Jul 21 06:17:52.599444 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.599421 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs9hb\" (UniqueName: \"kubernetes.io/projected/6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842-kube-api-access-qs9hb\") pod \"aws-ebs-csi-driver-node-2vkjf\" (UID: \"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.600695 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.600674 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmz64\" (UniqueName: \"kubernetes.io/projected/5763dac6-5176-4058-ae43-782e71cc676d-kube-api-access-mmz64\") pod \"tuned-ztv8x\" (UID: \"5763dac6-5176-4058-ae43-782e71cc676d\") " pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.679197 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.679080 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ks8rh" Jul 21 06:17:52.689872 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.689821 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-njwwp" Jul 21 06:17:52.701526 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.701501 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" Jul 21 06:17:52.707114 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.707094 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:17:52.714981 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.714962 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fkwwc" Jul 21 06:17:52.722512 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.722495 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nzgkk" Jul 21 06:17:52.729015 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.728998 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gzpql" Jul 21 06:17:52.736540 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.736523 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6dtks" Jul 21 06:17:52.742140 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.742123 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" Jul 21 06:17:52.921507 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.921476 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 06:17:52.992888 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.992802 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:52.992888 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.992860 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs\") pod \"network-metrics-daemon-l2867\" (UID: \"42908595-6e16-4688-b583-504934328eb7\") " pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:17:52.993124 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:52.992895 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzgpj\" (UniqueName: \"kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj\") pod \"network-check-target-5sldl\" (UID: \"a69c086c-f31f-416b-a386-0a3e3e458acb\") " pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:17:52.993124 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.992961 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 06:17:52.993124 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.993041 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 06:17:52.993124 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.993040 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret podName:2ee4a567-fe9e-4346-ac51-1d78dede57d1 nodeName:}" failed. No retries permitted until 2026-07-21 06:17:53.993015456 +0000 UTC m=+3.938654109 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret") pod "global-pull-secret-syncer-kwzdp" (UID: "2ee4a567-fe9e-4346-ac51-1d78dede57d1") : object "kube-system"/"original-pull-secret" not registered Jul 21 06:17:52.993124 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.993057 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:17:52.993124 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.993063 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 06:17:52.993124 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.993079 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzgpj for pod openshift-network-diagnostics/network-check-target-5sldl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:17:52.993124 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.993093 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs podName:42908595-6e16-4688-b583-504934328eb7 nodeName:}" failed. No retries permitted until 2026-07-21 06:17:53.993082255 +0000 UTC m=+3.938720891 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs") pod "network-metrics-daemon-l2867" (UID: "42908595-6e16-4688-b583-504934328eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:17:52.993531 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:52.993132 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj podName:a69c086c-f31f-416b-a386-0a3e3e458acb nodeName:}" failed. No retries permitted until 2026-07-21 06:17:53.993114454 +0000 UTC m=+3.938753094 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzgpj" (UniqueName: "kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj") pod "network-check-target-5sldl" (UID: "a69c086c-f31f-416b-a386-0a3e3e458acb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:17:53.415547 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:53.415463 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 06:12:53 +0000 UTC" deadline="2028-05-03 15:51:25.622705268 +0000 UTC" Jul 21 06:17:53.415547 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:53.415541 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15657h33m32.207168309s" Jul 21 06:17:53.532461 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:53.532376 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88a16c6f_e274_49dd_952e_065d7ba5b89c.slice/crio-5083ee9124edcb71e862d674b84ad7e3b22758a401b5ddd7b440840f061436c6 WatchSource:0}: Error finding container 5083ee9124edcb71e862d674b84ad7e3b22758a401b5ddd7b440840f061436c6: Status 404 returned error can't find the container with id 5083ee9124edcb71e862d674b84ad7e3b22758a401b5ddd7b440840f061436c6 Jul 21 06:17:53.535733 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:53.535706 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf0536e7_62ba_4202_99af_2d8c13d615f2.slice/crio-6e93eea6587792ccecf383383ba462bee1091f05059acaca52b5aac1f5768099 WatchSource:0}: Error finding container 6e93eea6587792ccecf383383ba462bee1091f05059acaca52b5aac1f5768099: Status 404 returned error can't find the container with id 6e93eea6587792ccecf383383ba462bee1091f05059acaca52b5aac1f5768099 Jul 21 06:17:53.536502 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:53.536467 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5763dac6_5176_4058_ae43_782e71cc676d.slice/crio-542a1ee4aa6223ab2c79cad62cd6134eef94d8632723e470175801a308b1ee04 WatchSource:0}: Error finding container 542a1ee4aa6223ab2c79cad62cd6134eef94d8632723e470175801a308b1ee04: Status 404 returned error can't find the container with id 542a1ee4aa6223ab2c79cad62cd6134eef94d8632723e470175801a308b1ee04 Jul 21 06:17:53.537575 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:53.537552 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd515ac_8a6a_4e1a_a0ad_b555677f3907.slice/crio-f082653f58412392aeab2db1182d5fefdcb1147a53cf0a35555e482b170f9d00 WatchSource:0}: Error finding container f082653f58412392aeab2db1182d5fefdcb1147a53cf0a35555e482b170f9d00: Status 404 returned error can't find the container with id f082653f58412392aeab2db1182d5fefdcb1147a53cf0a35555e482b170f9d00 Jul 21 06:17:53.538729 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:17:53.538445 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f6d1c44_4a9f_45d9_a3fd_64ec9ea21842.slice/crio-9d87ffec36fec93e75a5290da76c4bdf0e03b5074bd37805e0f3415234ab3cf1 WatchSource:0}: Error finding container 9d87ffec36fec93e75a5290da76c4bdf0e03b5074bd37805e0f3415234ab3cf1: Status 404 returned error can't find the container with id 9d87ffec36fec93e75a5290da76c4bdf0e03b5074bd37805e0f3415234ab3cf1 Jul 21 06:17:54.002405 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.001637 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs\") pod \"network-metrics-daemon-l2867\" (UID: \"42908595-6e16-4688-b583-504934328eb7\") " pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:17:54.002405 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:54.001913 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:17:54.002405 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:54.001989 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs podName:42908595-6e16-4688-b583-504934328eb7 nodeName:}" failed. No retries permitted until 2026-07-21 06:17:56.001965473 +0000 UTC m=+5.947604130 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs") pod "network-metrics-daemon-l2867" (UID: "42908595-6e16-4688-b583-504934328eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:17:54.002405 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:54.002022 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 06:17:54.002405 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:54.002041 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 06:17:54.002405 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:54.002053 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzgpj for pod openshift-network-diagnostics/network-check-target-5sldl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:17:54.002405 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:54.002102 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj podName:a69c086c-f31f-416b-a386-0a3e3e458acb nodeName:}" failed. No retries permitted until 2026-07-21 06:17:56.002086603 +0000 UTC m=+5.947725255 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzgpj" (UniqueName: "kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj") pod "network-check-target-5sldl" (UID: "a69c086c-f31f-416b-a386-0a3e3e458acb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:17:54.002405 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.001915 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzgpj\" (UniqueName: \"kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj\") pod \"network-check-target-5sldl\" (UID: \"a69c086c-f31f-416b-a386-0a3e3e458acb\") " pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:17:54.002405 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.002180 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:54.002405 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:54.002267 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 06:17:54.002405 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:54.002321 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret podName:2ee4a567-fe9e-4346-ac51-1d78dede57d1 nodeName:}" failed. No retries permitted until 2026-07-21 06:17:56.002309419 +0000 UTC m=+5.947948062 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret") pod "global-pull-secret-syncer-kwzdp" (UID: "2ee4a567-fe9e-4346-ac51-1d78dede57d1") : object "kube-system"/"original-pull-secret" not registered Jul 21 06:17:54.416418 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.416014 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 06:12:53 +0000 UTC" deadline="2028-04-18 09:46:03.85742645 +0000 UTC" Jul 21 06:17:54.416418 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.416054 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15291h28m9.441377839s" Jul 21 06:17:54.481393 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.481277 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:17:54.481564 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:54.481450 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:17:54.481564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.481555 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:54.481680 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:54.481639 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:17:54.483544 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.483389 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:17:54.483544 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:54.483511 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:17:54.512339 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.512118 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fkwwc" event={"ID":"88a16c6f-e274-49dd-952e-065d7ba5b89c","Type":"ContainerStarted","Data":"5083ee9124edcb71e862d674b84ad7e3b22758a401b5ddd7b440840f061436c6"} Jul 21 06:17:54.519967 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.519850 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" event={"ID":"add515ac-8a6a-4e1a-a0ad-b555677f3907","Type":"ContainerStarted","Data":"f082653f58412392aeab2db1182d5fefdcb1147a53cf0a35555e482b170f9d00"} Jul 21 06:17:54.542153 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.542118 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ks8rh" event={"ID":"512070fa-797a-4c81-92f3-0510a4c7cfbb","Type":"ContainerStarted","Data":"42cc2ed72300542959f3bdc8b1309dbece450ba14188c61171bb0e5def7a361f"} Jul 21 06:17:54.559400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.559340 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-159.ec2.internal" event={"ID":"0dad6f1e84139608ab928a8fbb218961","Type":"ContainerStarted","Data":"4496b278966f8e2fc8905048992912e4297510636bef1f5e0bb3735082a61854"} Jul 21 06:17:54.599717 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.599679 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nzgkk" event={"ID":"dd4adafa-b828-4ddc-8668-036cb8960d77","Type":"ContainerStarted","Data":"80f175486a8f14c644e52d0aae5f15795b938995409657b65a622156b1179d0a"} Jul 21 06:17:54.626141 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.625952 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-njwwp" event={"ID":"934f9315-0fc6-4f88-88fe-7b9b5cd6335d","Type":"ContainerStarted","Data":"1586d1f3fe069ce7bbb921fc28b6308b861f8b0b4fcab16a0d4ec9521ab1ef4c"} Jul 21 06:17:54.657447 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.657408 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gzpql" event={"ID":"cf0536e7-62ba-4202-99af-2d8c13d615f2","Type":"ContainerStarted","Data":"6e93eea6587792ccecf383383ba462bee1091f05059acaca52b5aac1f5768099"} Jul 21 06:17:54.668370 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.668267 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6dtks" event={"ID":"7b9eedde-6415-425e-9223-d74b2a3f74cb","Type":"ContainerStarted","Data":"a33148bbb8b7087759c61423dcdd1a0e08bd31f3f4e2eddd2ed6a2cca758c501"} Jul 21 06:17:54.676788 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.676753 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" event={"ID":"5763dac6-5176-4058-ae43-782e71cc676d","Type":"ContainerStarted","Data":"542a1ee4aa6223ab2c79cad62cd6134eef94d8632723e470175801a308b1ee04"} Jul 21 06:17:54.680924 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.680895 2563 generic.go:356] "Generic (PLEG): container finished" podID="e682834e4c02794e4aba5128ec0c7ad6" containerID="1cd0f29039a62248c1038f3d631a6e1e2033018b007574285bf97badfb8bc449" exitCode=0 Jul 21 06:17:54.681056 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.680940 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal" event={"ID":"e682834e4c02794e4aba5128ec0c7ad6","Type":"ContainerDied","Data":"1cd0f29039a62248c1038f3d631a6e1e2033018b007574285bf97badfb8bc449"} Jul 21 06:17:54.691778 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.691441 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" event={"ID":"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842","Type":"ContainerStarted","Data":"9d87ffec36fec93e75a5290da76c4bdf0e03b5074bd37805e0f3415234ab3cf1"} Jul 21 06:17:54.699774 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:54.699722 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-159.ec2.internal" podStartSLOduration=2.699703799 podStartE2EDuration="2.699703799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 06:17:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:17:54.577038668 +0000 UTC m=+4.522677327" watchObservedRunningTime="2026-07-21 06:17:54.699703799 +0000 UTC m=+4.645342458" Jul 21 06:17:55.714705 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:55.714666 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal" event={"ID":"e682834e4c02794e4aba5128ec0c7ad6","Type":"ContainerStarted","Data":"83f8d06edab5e72ff78d68f23182fc329a13eed4905dc153f30b2fa3b40cc190"} Jul 21 06:17:56.021845 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:56.021752 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:56.021845 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:56.021816 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs\") pod \"network-metrics-daemon-l2867\" (UID: \"42908595-6e16-4688-b583-504934328eb7\") " pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:17:56.022091 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:56.021853 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzgpj\" (UniqueName: \"kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj\") pod \"network-check-target-5sldl\" (UID: \"a69c086c-f31f-416b-a386-0a3e3e458acb\") " pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:17:56.022091 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:56.022010 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 06:17:56.022091 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:56.022030 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 06:17:56.022091 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:56.022043 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzgpj for pod openshift-network-diagnostics/network-check-target-5sldl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:17:56.022287 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:56.022110 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj podName:a69c086c-f31f-416b-a386-0a3e3e458acb nodeName:}" failed. No retries permitted until 2026-07-21 06:18:00.022088192 +0000 UTC m=+9.967726853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzgpj" (UniqueName: "kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj") pod "network-check-target-5sldl" (UID: "a69c086c-f31f-416b-a386-0a3e3e458acb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:17:56.022779 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:56.022755 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 06:17:56.022880 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:56.022845 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret podName:2ee4a567-fe9e-4346-ac51-1d78dede57d1 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:00.022800572 +0000 UTC m=+9.968439211 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret") pod "global-pull-secret-syncer-kwzdp" (UID: "2ee4a567-fe9e-4346-ac51-1d78dede57d1") : object "kube-system"/"original-pull-secret" not registered Jul 21 06:17:56.022974 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:56.022921 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:17:56.023028 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:56.022991 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs podName:42908595-6e16-4688-b583-504934328eb7 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:00.02297565 +0000 UTC m=+9.968614290 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs") pod "network-metrics-daemon-l2867" (UID: "42908595-6e16-4688-b583-504934328eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:17:56.482348 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:56.481430 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:56.482348 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:56.481573 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:17:56.482348 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:56.481613 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:17:56.482348 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:56.481715 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:17:56.482348 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:56.481771 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:17:56.482348 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:56.481901 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:17:58.481900 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:58.481370 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:17:58.481900 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:58.481371 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:17:58.481900 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:58.481518 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:17:58.481900 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:17:58.481587 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:17:58.481900 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:58.481699 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:17:58.481900 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:17:58.481772 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:00.056413 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:00.056137 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs\") pod \"network-metrics-daemon-l2867\" (UID: \"42908595-6e16-4688-b583-504934328eb7\") " pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:00.056413 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:00.056189 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzgpj\" (UniqueName: \"kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj\") pod \"network-check-target-5sldl\" (UID: \"a69c086c-f31f-416b-a386-0a3e3e458acb\") " pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:00.056413 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:00.056237 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:00.056413 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:00.056313 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:18:00.056413 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:00.056339 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 06:18:00.056413 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:00.056353 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 06:18:00.056413 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:00.056362 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 06:18:00.056413 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:00.056376 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzgpj for pod openshift-network-diagnostics/network-check-target-5sldl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:18:00.056413 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:00.056392 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs podName:42908595-6e16-4688-b583-504934328eb7 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:08.056368413 +0000 UTC m=+18.002007065 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs") pod "network-metrics-daemon-l2867" (UID: "42908595-6e16-4688-b583-504934328eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:18:00.056413 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:00.056412 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret podName:2ee4a567-fe9e-4346-ac51-1d78dede57d1 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:08.056401921 +0000 UTC m=+18.002040559 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret") pod "global-pull-secret-syncer-kwzdp" (UID: "2ee4a567-fe9e-4346-ac51-1d78dede57d1") : object "kube-system"/"original-pull-secret" not registered Jul 21 06:18:00.056413 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:00.056428 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj podName:a69c086c-f31f-416b-a386-0a3e3e458acb nodeName:}" failed. No retries permitted until 2026-07-21 06:18:08.056418281 +0000 UTC m=+18.002056940 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzgpj" (UniqueName: "kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj") pod "network-check-target-5sldl" (UID: "a69c086c-f31f-416b-a386-0a3e3e458acb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:18:00.483551 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:00.482692 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:00.483551 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:00.482829 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:00.483551 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:00.483208 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:00.483551 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:00.483336 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:18:00.483551 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:00.483386 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:00.483551 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:00.483458 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:02.484002 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:02.483193 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:02.484002 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:02.483326 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:02.484002 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:02.483708 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:02.484002 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:02.483792 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:02.484002 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:02.483864 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:02.484002 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:02.483949 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:18:04.480190 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:04.480152 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:04.480658 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:04.480152 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:04.480658 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:04.480313 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:04.480658 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:04.480152 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:04.480658 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:04.480384 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:18:04.480658 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:04.480439 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:06.481138 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:06.481105 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:06.481593 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:06.481144 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:06.481593 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:06.481105 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:06.481593 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:06.481245 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:06.481593 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:06.481310 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:06.481593 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:06.481395 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:18:08.117442 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:08.117398 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs\") pod \"network-metrics-daemon-l2867\" (UID: \"42908595-6e16-4688-b583-504934328eb7\") " pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:08.117931 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:08.117459 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzgpj\" (UniqueName: \"kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj\") pod \"network-check-target-5sldl\" (UID: \"a69c086c-f31f-416b-a386-0a3e3e458acb\") " pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:08.117931 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:08.117512 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:08.117931 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:08.117556 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:18:08.117931 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:08.117615 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 06:18:08.117931 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:08.117637 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs podName:42908595-6e16-4688-b583-504934328eb7 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:24.117619957 +0000 UTC m=+34.063258596 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs") pod "network-metrics-daemon-l2867" (UID: "42908595-6e16-4688-b583-504934328eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:18:08.117931 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:08.117641 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 06:18:08.117931 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:08.117665 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 06:18:08.117931 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:08.117666 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret podName:2ee4a567-fe9e-4346-ac51-1d78dede57d1 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:24.117651206 +0000 UTC m=+34.063289842 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret") pod "global-pull-secret-syncer-kwzdp" (UID: "2ee4a567-fe9e-4346-ac51-1d78dede57d1") : object "kube-system"/"original-pull-secret" not registered Jul 21 06:18:08.117931 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:08.117679 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzgpj for pod openshift-network-diagnostics/network-check-target-5sldl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:18:08.117931 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:08.117722 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj podName:a69c086c-f31f-416b-a386-0a3e3e458acb nodeName:}" failed. No retries permitted until 2026-07-21 06:18:24.117712223 +0000 UTC m=+34.063350874 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzgpj" (UniqueName: "kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj") pod "network-check-target-5sldl" (UID: "a69c086c-f31f-416b-a386-0a3e3e458acb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:18:08.480844 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:08.480761 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:08.481000 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:08.480771 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:08.481000 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:08.480881 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:08.481000 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:08.480774 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:08.481000 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:08.480965 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:08.481194 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:08.481068 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:18:10.481592 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:10.481528 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:10.481592 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:10.481590 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:10.481975 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:10.481661 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:10.481975 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:10.481724 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:10.481975 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:10.481754 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:10.481975 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:10.481844 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:18:12.480787 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:12.480755 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:12.481315 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:12.480755 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:12.481315 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:12.480893 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:12.481315 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:12.480899 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:18:12.481315 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:12.480988 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:12.481315 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:12.481088 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:13.753692 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.753301 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6dtks" event={"ID":"7b9eedde-6415-425e-9223-d74b2a3f74cb","Type":"ContainerStarted","Data":"ef3ad171e995bf43889f8355ccb2ef182c44b8d25948e62e53943e0c77cbefb9"} Jul 21 06:18:13.754933 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.754907 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" event={"ID":"5763dac6-5176-4058-ae43-782e71cc676d","Type":"ContainerStarted","Data":"beb335e5d02f07edf54339c1b00754b8fdbd3a6c2206926c546c94254ec2d69b"} Jul 21 06:18:13.756351 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.756330 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" event={"ID":"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842","Type":"ContainerStarted","Data":"734ecfafe843e372c49ef12a951c47a7c704fa697bbc5405f546210c53913623"} Jul 21 06:18:13.757853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.757828 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fkwwc" event={"ID":"88a16c6f-e274-49dd-952e-065d7ba5b89c","Type":"ContainerStarted","Data":"4885e8af50f18277a4cc1202da4bc45d113de1ae32c4c423a7bf19c95fd6b88a"} Jul 21 06:18:13.760133 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.760111 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" event={"ID":"add515ac-8a6a-4e1a-a0ad-b555677f3907","Type":"ContainerStarted","Data":"2c4831950b4cb68d0f965ddb85f4c4d5ee901a7e19f4f2707edc6b3a467da081"} Jul 21 06:18:13.760245 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.760187 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" event={"ID":"add515ac-8a6a-4e1a-a0ad-b555677f3907","Type":"ContainerStarted","Data":"0561bcdd3799d9876fad5df0eb2289cef9f3ddee9953da17826c4d51993d065a"} Jul 21 06:18:13.760313 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.760266 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" event={"ID":"add515ac-8a6a-4e1a-a0ad-b555677f3907","Type":"ContainerStarted","Data":"14ed773e67bfcc8f69c877ce8502a8d7b9ee08432661df0fc028fcad2d63e122"} Jul 21 06:18:13.762690 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.762670 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ks8rh" event={"ID":"512070fa-797a-4c81-92f3-0510a4c7cfbb","Type":"ContainerStarted","Data":"2c002700251de3a6236910a89e460a455c9ba7378bd76937dadea9053dd4d565"} Jul 21 06:18:13.764157 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.764127 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nzgkk" event={"ID":"dd4adafa-b828-4ddc-8668-036cb8960d77","Type":"ContainerStarted","Data":"2be974550cd280643f7ef1c30ee933b6d6d992688768bcccc98199b907a797b3"} Jul 21 06:18:13.768978 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.768929 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6dtks" podStartSLOduration=3.07674944 podStartE2EDuration="22.768914414s" totalImagesPullingTime="19.692164974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 06:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:18:13.76864557 +0000 UTC m=+23.714284230" watchObservedRunningTime="2026-07-21 06:18:13.768914414 +0000 UTC m=+23.714553063" Jul 21 06:18:13.769073 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.769054 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-159.ec2.internal" podStartSLOduration=21.769049111 podStartE2EDuration="21.769049111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 06:17:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:17:55.731083967 +0000 UTC m=+5.676722630" watchObservedRunningTime="2026-07-21 06:18:13.769049111 +0000 UTC m=+23.714687768" Jul 21 06:18:13.769136 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.769113 2563 generic.go:356] "Generic (PLEG): container finished" podID="cf0536e7-62ba-4202-99af-2d8c13d615f2" containerID="000df9b22411a6350831992bfbd0210f92ddcf51e55e01bc808d8fd6e6e5e36e" exitCode=0 Jul 21 06:18:13.769189 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.769146 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gzpql" event={"ID":"cf0536e7-62ba-4202-99af-2d8c13d615f2","Type":"ContainerDied","Data":"000df9b22411a6350831992bfbd0210f92ddcf51e55e01bc808d8fd6e6e5e36e"} Jul 21 06:18:13.785697 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.785662 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ks8rh" podStartSLOduration=3.086600627 podStartE2EDuration="22.785650026s" totalImagesPullingTime="19.699049399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 06:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:18:13.785138907 +0000 UTC m=+23.730777565" watchObservedRunningTime="2026-07-21 06:18:13.785650026 +0000 UTC m=+23.731288683" Jul 21 06:18:13.803255 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.803198 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fkwwc" podStartSLOduration=3.041844137 podStartE2EDuration="22.80318604s" totalImagesPullingTime="19.761341903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 06:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:18:13.802734929 +0000 UTC m=+23.748373586" watchObservedRunningTime="2026-07-21 06:18:13.80318604 +0000 UTC m=+23.748824696" Jul 21 06:18:13.817562 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.817527 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nzgkk" podStartSLOduration=3.144044248 podStartE2EDuration="22.81751417s" totalImagesPullingTime="19.673469922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:18:13.816812596 +0000 UTC m=+23.762451244" watchObservedRunningTime="2026-07-21 06:18:13.81751417 +0000 UTC m=+23.763152827" Jul 21 06:18:13.834954 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:13.834911 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ztv8x" podStartSLOduration=3.14624577 podStartE2EDuration="22.834900161s" totalImagesPullingTime="19.688654391s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 06:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:18:13.834786932 +0000 UTC m=+23.780425588" watchObservedRunningTime="2026-07-21 06:18:13.834900161 +0000 UTC m=+23.780538818" Jul 21 06:18:14.258901 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:14.258875 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nzgkk" Jul 21 06:18:14.259577 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:14.259552 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nzgkk" Jul 21 06:18:14.480939 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:14.480661 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:14.480939 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:14.480682 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:14.480939 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:14.480772 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:14.480939 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:14.480804 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:14.480939 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:14.480839 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:14.480939 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:14.480912 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:18:14.774442 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:14.774411 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-njwwp" event={"ID":"934f9315-0fc6-4f88-88fe-7b9b5cd6335d","Type":"ContainerStarted","Data":"43e52335ae47bbec50ab6eca3da2961ab0aac3a8b88799fed9c27062b46634fc"} Jul 21 06:18:14.778529 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:14.778499 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" event={"ID":"add515ac-8a6a-4e1a-a0ad-b555677f3907","Type":"ContainerStarted","Data":"216bf25acc00ea1749a373540d38e63d6ccc710f6b1c92224abe0e3b0457ff60"} Jul 21 06:18:14.778649 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:14.778569 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" event={"ID":"add515ac-8a6a-4e1a-a0ad-b555677f3907","Type":"ContainerStarted","Data":"10139fbe0851a7afd59b3ac2b74d6d2eb7f4946b1347a1352475f1f1ccaf1579"} Jul 21 06:18:14.778649 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:14.778606 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" event={"ID":"add515ac-8a6a-4e1a-a0ad-b555677f3907","Type":"ContainerStarted","Data":"e0681d9ad8cd80b59d13bb0b071522dc8ad6a3144e7402fcde01354f13f39d13"} Jul 21 06:18:14.787446 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:14.787423 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nzgkk" Jul 21 06:18:14.788002 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:14.787975 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nzgkk" Jul 21 06:18:14.792185 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:14.792141 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-njwwp" podStartSLOduration=4.100391262 podStartE2EDuration="23.792126081s" totalImagesPullingTime="19.691734819s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 06:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:18:14.791736722 +0000 UTC m=+24.737375380" watchObservedRunningTime="2026-07-21 06:18:14.792126081 +0000 UTC m=+24.737764746" Jul 21 06:18:14.902674 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:14.902654 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 06:18:15.433116 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:15.433007 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T06:18:14.902669395Z","UUID":"58b1507e-2864-471c-b3a5-849f2ebfa89b","Handler":null,"Name":"","Endpoint":""} Jul 21 06:18:15.434784 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:15.434763 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 06:18:15.434917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:15.434804 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 06:18:15.783067 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:15.782993 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" event={"ID":"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842","Type":"ContainerStarted","Data":"072194d156ed5d6e7250538bb814e5e11cc8d4e38804f3831a821b44e2432193"} Jul 21 06:18:16.480293 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:16.480186 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:16.480293 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:16.480239 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:16.480293 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:16.480186 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:16.480543 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:16.480326 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:16.480543 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:16.480384 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:16.480543 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:16.480444 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:18:16.787661 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:16.787574 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" event={"ID":"6f6d1c44-4a9f-45d9-a3fd-64ec9ea21842","Type":"ContainerStarted","Data":"59a0d6cc71c3c0879380c6734924164b6fdf6613fea7d7d02c31879db0437ce4"} Jul 21 06:18:16.791409 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:16.791380 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" event={"ID":"add515ac-8a6a-4e1a-a0ad-b555677f3907","Type":"ContainerStarted","Data":"3ef0f18d4c3b9b074ff6a8a3bdb38b0365832ccf32b59be45e91162ba87b55b1"} Jul 21 06:18:16.804709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:16.804664 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2vkjf" podStartSLOduration=3.780906548 podStartE2EDuration="25.804650344s" totalImagesPullingTime="22.023743796s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 06:17:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:18:16.80443625 +0000 UTC m=+26.750074908" watchObservedRunningTime="2026-07-21 06:18:16.804650344 +0000 UTC m=+26.750289001" Jul 21 06:18:18.481069 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:18.480842 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:18.481774 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:18.480842 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:18.481774 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:18.481105 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:18.481774 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:18.481192 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:18.481774 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:18.480916 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:18.481774 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:18.481311 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:18:18.798857 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:18.798779 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" event={"ID":"add515ac-8a6a-4e1a-a0ad-b555677f3907","Type":"ContainerStarted","Data":"85fab1f365f5a69fabfad27cc69c14947effb46a32d5ac8b20eaf44aedd41abb"} Jul 21 06:18:18.799027 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:18.799010 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:18:18.799095 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:18.799076 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:18:18.800688 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:18.800665 2563 generic.go:356] "Generic (PLEG): container finished" podID="cf0536e7-62ba-4202-99af-2d8c13d615f2" containerID="7cf276d21730fc44bb6f450dbd947e0c7b002a5562972bacf77a9561c56ed206" exitCode=0 Jul 21 06:18:18.800807 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:18.800697 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gzpql" event={"ID":"cf0536e7-62ba-4202-99af-2d8c13d615f2","Type":"ContainerDied","Data":"7cf276d21730fc44bb6f450dbd947e0c7b002a5562972bacf77a9561c56ed206"} Jul 21 06:18:18.815130 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:18.815109 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:18:18.825476 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:18.825437 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" podStartSLOduration=8.063104497 podStartE2EDuration="27.825425002s" totalImagesPullingTime="19.762320505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 06:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:18:18.825084749 +0000 UTC m=+28.770723406" watchObservedRunningTime="2026-07-21 06:18:18.825425002 +0000 UTC m=+28.771063659" Jul 21 06:18:19.803300 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:19.803091 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:18:19.817658 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:19.817639 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:18:20.185157 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:20.185130 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kwzdp"] Jul 21 06:18:20.185350 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:20.185282 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:20.185390 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:20.185361 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:20.188183 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:20.188141 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5sldl"] Jul 21 06:18:20.188337 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:20.188264 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:20.188420 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:20.188343 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:20.188818 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:20.188791 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l2867"] Jul 21 06:18:20.188942 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:20.188902 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:20.189029 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:20.189010 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:18:20.806590 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:20.806553 2563 generic.go:356] "Generic (PLEG): container finished" podID="cf0536e7-62ba-4202-99af-2d8c13d615f2" containerID="1fdd5e542ddcd026acb09820f6a6359efe0dffd6a0411b531d12989be29d811b" exitCode=0 Jul 21 06:18:20.807029 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:20.806639 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gzpql" event={"ID":"cf0536e7-62ba-4202-99af-2d8c13d615f2","Type":"ContainerDied","Data":"1fdd5e542ddcd026acb09820f6a6359efe0dffd6a0411b531d12989be29d811b"} Jul 21 06:18:21.480238 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:21.480189 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:21.480401 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:21.480308 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:22.480804 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:22.480610 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:22.481168 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:22.480610 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:22.481168 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:22.480892 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:18:22.481168 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:22.480949 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:22.813354 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:22.813320 2563 generic.go:356] "Generic (PLEG): container finished" podID="cf0536e7-62ba-4202-99af-2d8c13d615f2" containerID="7e5f8430fb400d4cd81a64d6c78dbdbca90a5e13f3cf4fcc085eb810fa1f7705" exitCode=0 Jul 21 06:18:22.813499 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:22.813362 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gzpql" event={"ID":"cf0536e7-62ba-4202-99af-2d8c13d615f2","Type":"ContainerDied","Data":"7e5f8430fb400d4cd81a64d6c78dbdbca90a5e13f3cf4fcc085eb810fa1f7705"} Jul 21 06:18:23.480554 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:23.480515 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:23.480744 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:23.480655 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:24.143340 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:24.143304 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:24.143742 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:24.143367 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs\") pod \"network-metrics-daemon-l2867\" (UID: \"42908595-6e16-4688-b583-504934328eb7\") " pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:24.143742 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:24.143403 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzgpj\" (UniqueName: \"kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj\") pod \"network-check-target-5sldl\" (UID: \"a69c086c-f31f-416b-a386-0a3e3e458acb\") " pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:24.143742 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:24.143449 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 06:18:24.143742 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:24.143504 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret podName:2ee4a567-fe9e-4346-ac51-1d78dede57d1 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:56.143488135 +0000 UTC m=+66.089126774 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret") pod "global-pull-secret-syncer-kwzdp" (UID: "2ee4a567-fe9e-4346-ac51-1d78dede57d1") : object "kube-system"/"original-pull-secret" not registered Jul 21 06:18:24.143742 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:24.143531 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 06:18:24.143742 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:24.143550 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 06:18:24.143742 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:24.143562 2563 projected.go:196] Error preparing data for projected volume kube-api-access-tzgpj for pod openshift-network-diagnostics/network-check-target-5sldl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:18:24.143742 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:24.143607 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj podName:a69c086c-f31f-416b-a386-0a3e3e458acb nodeName:}" failed. No retries permitted until 2026-07-21 06:18:56.143591931 +0000 UTC m=+66.089230569 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzgpj" (UniqueName: "kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj") pod "network-check-target-5sldl" (UID: "a69c086c-f31f-416b-a386-0a3e3e458acb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 06:18:24.143742 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:24.143666 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:18:24.143742 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:24.143700 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs podName:42908595-6e16-4688-b583-504934328eb7 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:56.143688526 +0000 UTC m=+66.089327166 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs") pod "network-metrics-daemon-l2867" (UID: "42908595-6e16-4688-b583-504934328eb7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 06:18:24.480474 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:24.480443 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:24.480474 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:24.480462 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:24.480820 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:24.480573 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l2867" podUID="42908595-6e16-4688-b583-504934328eb7" Jul 21 06:18:24.480820 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:24.480712 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5sldl" podUID="a69c086c-f31f-416b-a386-0a3e3e458acb" Jul 21 06:18:25.480711 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.480676 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:25.481203 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:25.480821 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kwzdp" podUID="2ee4a567-fe9e-4346-ac51-1d78dede57d1" Jul 21 06:18:25.857322 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.857291 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-159.ec2.internal" event="NodeReady" Jul 21 06:18:25.857498 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.857423 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 06:18:25.896134 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.896105 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2gh49"] Jul 21 06:18:25.898843 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.898815 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:25.899270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.899246 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-w6zct"] Jul 21 06:18:25.900784 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.900765 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w6zct" Jul 21 06:18:25.901727 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.901702 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 06:18:25.901941 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.901927 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 06:18:25.902135 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.902120 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5sp2g\"" Jul 21 06:18:25.903552 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.903534 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rjnwl\"" Jul 21 06:18:25.904003 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.903775 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 06:18:25.904003 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.903788 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 06:18:25.904003 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.903796 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 06:18:25.910844 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.910813 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2gh49"] Jul 21 06:18:25.911638 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:25.911617 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w6zct"] Jul 21 06:18:26.058276 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.058192 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:26.058276 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.058237 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gghfv\" (UniqueName: \"kubernetes.io/projected/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-kube-api-access-gghfv\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:26.058479 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.058313 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c253498b-ea18-4b4d-b21d-116381a32d7f-cert\") pod \"ingress-canary-w6zct\" (UID: \"c253498b-ea18-4b4d-b21d-116381a32d7f\") " pod="openshift-ingress-canary/ingress-canary-w6zct" Jul 21 06:18:26.058479 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.058366 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-config-volume\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:26.058479 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.058421 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmlhb\" (UniqueName: \"kubernetes.io/projected/c253498b-ea18-4b4d-b21d-116381a32d7f-kube-api-access-qmlhb\") pod \"ingress-canary-w6zct\" (UID: \"c253498b-ea18-4b4d-b21d-116381a32d7f\") " pod="openshift-ingress-canary/ingress-canary-w6zct" Jul 21 06:18:26.058479 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.058477 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-tmp-dir\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:26.159496 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.159459 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-tmp-dir\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:26.159666 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.159525 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:26.159666 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.159548 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gghfv\" (UniqueName: \"kubernetes.io/projected/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-kube-api-access-gghfv\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:26.159666 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.159568 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c253498b-ea18-4b4d-b21d-116381a32d7f-cert\") pod \"ingress-canary-w6zct\" (UID: \"c253498b-ea18-4b4d-b21d-116381a32d7f\") " pod="openshift-ingress-canary/ingress-canary-w6zct" Jul 21 06:18:26.159666 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.159596 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-config-volume\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:26.159666 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.159634 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmlhb\" (UniqueName: \"kubernetes.io/projected/c253498b-ea18-4b4d-b21d-116381a32d7f-kube-api-access-qmlhb\") pod \"ingress-canary-w6zct\" (UID: \"c253498b-ea18-4b4d-b21d-116381a32d7f\") " pod="openshift-ingress-canary/ingress-canary-w6zct" Jul 21 06:18:26.159666 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:26.159648 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 06:18:26.159943 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:26.159725 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls podName:8b9dd21a-f6c5-478c-8dc5-024a0fb145f2 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:26.659703042 +0000 UTC m=+36.605341700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls") pod "dns-default-2gh49" (UID: "8b9dd21a-f6c5-478c-8dc5-024a0fb145f2") : secret "dns-default-metrics-tls" not found Jul 21 06:18:26.159943 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.159838 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-tmp-dir\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:26.160269 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.160249 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-config-volume\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:26.164445 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.164421 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c253498b-ea18-4b4d-b21d-116381a32d7f-cert\") pod \"ingress-canary-w6zct\" (UID: \"c253498b-ea18-4b4d-b21d-116381a32d7f\") " pod="openshift-ingress-canary/ingress-canary-w6zct" Jul 21 06:18:26.168942 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.168919 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gghfv\" (UniqueName: \"kubernetes.io/projected/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-kube-api-access-gghfv\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:26.169627 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.169602 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmlhb\" (UniqueName: \"kubernetes.io/projected/c253498b-ea18-4b4d-b21d-116381a32d7f-kube-api-access-qmlhb\") pod \"ingress-canary-w6zct\" (UID: \"c253498b-ea18-4b4d-b21d-116381a32d7f\") " pod="openshift-ingress-canary/ingress-canary-w6zct" Jul 21 06:18:26.218503 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.218475 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w6zct" Jul 21 06:18:26.480325 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.480288 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:26.480325 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.480328 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:26.485239 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.485196 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 06:18:26.485239 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.485201 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 06:18:26.485935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.485289 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 06:18:26.485935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.485504 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8rpb4\"" Jul 21 06:18:26.485935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.485706 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8684r\"" Jul 21 06:18:26.664593 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.664034 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:26.664593 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:26.664139 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 06:18:26.664593 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:26.664207 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls podName:8b9dd21a-f6c5-478c-8dc5-024a0fb145f2 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:27.664183419 +0000 UTC m=+37.609822068 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls") pod "dns-default-2gh49" (UID: "8b9dd21a-f6c5-478c-8dc5-024a0fb145f2") : secret "dns-default-metrics-tls" not found Jul 21 06:18:26.978719 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:26.978523 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w6zct"] Jul 21 06:18:26.983185 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:18:26.983161 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc253498b_ea18_4b4d_b21d_116381a32d7f.slice/crio-f44ea13f8f86cbeeac7b2a81c67a3c3f96ea02898e2c64c0a79a3855531a997b WatchSource:0}: Error finding container f44ea13f8f86cbeeac7b2a81c67a3c3f96ea02898e2c64c0a79a3855531a997b: Status 404 returned error can't find the container with id f44ea13f8f86cbeeac7b2a81c67a3c3f96ea02898e2c64c0a79a3855531a997b Jul 21 06:18:27.480542 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:27.480472 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:27.484090 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:27.484060 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 06:18:27.672111 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:27.672075 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:27.672649 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:27.672241 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 06:18:27.672649 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:27.672294 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls podName:8b9dd21a-f6c5-478c-8dc5-024a0fb145f2 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:29.672279712 +0000 UTC m=+39.617918346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls") pod "dns-default-2gh49" (UID: "8b9dd21a-f6c5-478c-8dc5-024a0fb145f2") : secret "dns-default-metrics-tls" not found Jul 21 06:18:27.828683 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:27.828650 2563 generic.go:356] "Generic (PLEG): container finished" podID="cf0536e7-62ba-4202-99af-2d8c13d615f2" containerID="7e5a7d4f9c9e7fbb39ba7790617e70920f939292f71ea0f39dd74c9a0af882c5" exitCode=0 Jul 21 06:18:27.828863 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:27.828748 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gzpql" event={"ID":"cf0536e7-62ba-4202-99af-2d8c13d615f2","Type":"ContainerDied","Data":"7e5a7d4f9c9e7fbb39ba7790617e70920f939292f71ea0f39dd74c9a0af882c5"} Jul 21 06:18:27.829879 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:27.829855 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w6zct" event={"ID":"c253498b-ea18-4b4d-b21d-116381a32d7f","Type":"ContainerStarted","Data":"f44ea13f8f86cbeeac7b2a81c67a3c3f96ea02898e2c64c0a79a3855531a997b"} Jul 21 06:18:28.836454 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:28.836421 2563 generic.go:356] "Generic (PLEG): container finished" podID="cf0536e7-62ba-4202-99af-2d8c13d615f2" containerID="c90553f831598dab23d43f6bc40376bcc6c1df13cca705426f5999cd5d74aa25" exitCode=0 Jul 21 06:18:28.836879 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:28.836497 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gzpql" event={"ID":"cf0536e7-62ba-4202-99af-2d8c13d615f2","Type":"ContainerDied","Data":"c90553f831598dab23d43f6bc40376bcc6c1df13cca705426f5999cd5d74aa25"} Jul 21 06:18:29.689190 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:29.688958 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:29.689434 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:29.689119 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 06:18:29.689434 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:29.689309 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls podName:8b9dd21a-f6c5-478c-8dc5-024a0fb145f2 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:33.689291984 +0000 UTC m=+43.634930633 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls") pod "dns-default-2gh49" (UID: "8b9dd21a-f6c5-478c-8dc5-024a0fb145f2") : secret "dns-default-metrics-tls" not found Jul 21 06:18:29.841799 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:29.841763 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gzpql" event={"ID":"cf0536e7-62ba-4202-99af-2d8c13d615f2","Type":"ContainerStarted","Data":"b32e990a3939404ac2adc53433e6a28a75111d7bd005d55b173fed4272911d98"} Jul 21 06:18:29.843123 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:29.843100 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w6zct" event={"ID":"c253498b-ea18-4b4d-b21d-116381a32d7f","Type":"ContainerStarted","Data":"7f68fa8221f421afd57661eaecb2fbd8e00caeac5ad33a507cae608193847179"} Jul 21 06:18:29.865448 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:29.865394 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gzpql" podStartSLOduration=8.771489811 podStartE2EDuration="38.865376598s" totalImagesPullingTime="30.093886787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 06:17:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:18:29.863636159 +0000 UTC m=+39.809274817" watchObservedRunningTime="2026-07-21 06:18:29.865376598 +0000 UTC m=+39.811015258" Jul 21 06:18:29.878662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:29.878615 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-w6zct" podStartSLOduration=2.7486232619999997 podStartE2EDuration="4.878597776s" totalImagesPullingTime="2.129974514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:18:29.877712131 +0000 UTC m=+39.823350789" watchObservedRunningTime="2026-07-21 06:18:29.878597776 +0000 UTC m=+39.824236435" Jul 21 06:18:33.717343 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:33.717307 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:33.717790 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:33.717471 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 06:18:33.717790 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:33.717535 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls podName:8b9dd21a-f6c5-478c-8dc5-024a0fb145f2 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:41.717517881 +0000 UTC m=+51.663156515 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls") pod "dns-default-2gh49" (UID: "8b9dd21a-f6c5-478c-8dc5-024a0fb145f2") : secret "dns-default-metrics-tls" not found Jul 21 06:18:34.451724 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:34.451690 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9prh8"] Jul 21 06:18:34.478781 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:34.478759 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9prh8"] Jul 21 06:18:34.478919 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:34.478854 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9prh8" Jul 21 06:18:34.481740 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:34.481706 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 21 06:18:34.481884 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:34.481761 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zrc27\"" Jul 21 06:18:34.483132 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:34.483114 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 21 06:18:34.624225 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:34.624188 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfnk6\" (UniqueName: \"kubernetes.io/projected/e7478004-cb1c-4941-a111-d7e5bc4e308d-kube-api-access-xfnk6\") pod \"migrator-7cc9c7ffd7-9prh8\" (UID: \"e7478004-cb1c-4941-a111-d7e5bc4e308d\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9prh8" Jul 21 06:18:34.724722 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:34.724634 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfnk6\" (UniqueName: \"kubernetes.io/projected/e7478004-cb1c-4941-a111-d7e5bc4e308d-kube-api-access-xfnk6\") pod \"migrator-7cc9c7ffd7-9prh8\" (UID: \"e7478004-cb1c-4941-a111-d7e5bc4e308d\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9prh8" Jul 21 06:18:34.735272 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:34.735246 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfnk6\" (UniqueName: \"kubernetes.io/projected/e7478004-cb1c-4941-a111-d7e5bc4e308d-kube-api-access-xfnk6\") pod \"migrator-7cc9c7ffd7-9prh8\" (UID: \"e7478004-cb1c-4941-a111-d7e5bc4e308d\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9prh8" Jul 21 06:18:34.788010 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:34.787987 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9prh8" Jul 21 06:18:34.927943 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:34.927911 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9prh8"] Jul 21 06:18:34.942761 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:18:34.942728 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7478004_cb1c_4941_a111_d7e5bc4e308d.slice/crio-faca48bc6272fa579d7e49fab55267783700ce2e758522fea37f4d21d507a033 WatchSource:0}: Error finding container faca48bc6272fa579d7e49fab55267783700ce2e758522fea37f4d21d507a033: Status 404 returned error can't find the container with id faca48bc6272fa579d7e49fab55267783700ce2e758522fea37f4d21d507a033 Jul 21 06:18:35.858055 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:35.858023 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9prh8" event={"ID":"e7478004-cb1c-4941-a111-d7e5bc4e308d","Type":"ContainerStarted","Data":"faca48bc6272fa579d7e49fab55267783700ce2e758522fea37f4d21d507a033"} Jul 21 06:18:36.861631 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:36.861598 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9prh8" event={"ID":"e7478004-cb1c-4941-a111-d7e5bc4e308d","Type":"ContainerStarted","Data":"56bb9a0e3f0eba029a115c023f38d034abd691b071bba66d784c10d78c03f6c8"} Jul 21 06:18:37.865891 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:37.865853 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9prh8" event={"ID":"e7478004-cb1c-4941-a111-d7e5bc4e308d","Type":"ContainerStarted","Data":"4f0a6bc433bdcb39518c1717bba60c984492fca7181ed27b2e3c1f702015ae77"} Jul 21 06:18:37.881682 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:37.881637 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-9prh8" podStartSLOduration=2.089641759 podStartE2EDuration="3.881622421s" totalImagesPullingTime="1.791980662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:18:37.880638068 +0000 UTC m=+47.826276734" watchObservedRunningTime="2026-07-21 06:18:37.881622421 +0000 UTC m=+47.827261056" Jul 21 06:18:41.773144 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:41.773103 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:41.773593 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:41.773240 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 06:18:41.773593 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:18:41.773292 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls podName:8b9dd21a-f6c5-478c-8dc5-024a0fb145f2 nodeName:}" failed. No retries permitted until 2026-07-21 06:18:57.773279294 +0000 UTC m=+67.718917929 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls") pod "dns-default-2gh49" (UID: "8b9dd21a-f6c5-478c-8dc5-024a0fb145f2") : secret "dns-default-metrics-tls" not found Jul 21 06:18:51.823286 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:51.823255 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8qbzm" Jul 21 06:18:56.170544 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.170502 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs\") pod \"network-metrics-daemon-l2867\" (UID: \"42908595-6e16-4688-b583-504934328eb7\") " pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:56.170544 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.170551 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzgpj\" (UniqueName: \"kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj\") pod \"network-check-target-5sldl\" (UID: \"a69c086c-f31f-416b-a386-0a3e3e458acb\") " pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:56.171113 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.170595 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:56.173772 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.173743 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 06:18:56.175089 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.175072 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 06:18:56.175198 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.175076 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 06:18:56.183859 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.183840 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ee4a567-fe9e-4346-ac51-1d78dede57d1-original-pull-secret\") pod \"global-pull-secret-syncer-kwzdp\" (UID: \"2ee4a567-fe9e-4346-ac51-1d78dede57d1\") " pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:56.183942 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.183916 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42908595-6e16-4688-b583-504934328eb7-metrics-certs\") pod \"network-metrics-daemon-l2867\" (UID: \"42908595-6e16-4688-b583-504934328eb7\") " pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:56.184172 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.184155 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 06:18:56.194803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.194781 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzgpj\" (UniqueName: \"kubernetes.io/projected/a69c086c-f31f-416b-a386-0a3e3e458acb-kube-api-access-tzgpj\") pod \"network-check-target-5sldl\" (UID: \"a69c086c-f31f-416b-a386-0a3e3e458acb\") " pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:56.195526 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.195514 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8rpb4\"" Jul 21 06:18:56.201268 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.201252 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8684r\"" Jul 21 06:18:56.203363 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.203348 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l2867" Jul 21 06:18:56.208893 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.208880 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:18:56.291582 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.291308 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kwzdp" Jul 21 06:18:56.350932 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.350854 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5sldl"] Jul 21 06:18:56.356024 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:18:56.355912 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69c086c_f31f_416b_a386_0a3e3e458acb.slice/crio-b638c9529b65de59315c638b67993118d518367477093f25b4673677dab073cb WatchSource:0}: Error finding container b638c9529b65de59315c638b67993118d518367477093f25b4673677dab073cb: Status 404 returned error can't find the container with id b638c9529b65de59315c638b67993118d518367477093f25b4673677dab073cb Jul 21 06:18:56.370917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.370881 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l2867"] Jul 21 06:18:56.375759 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:18:56.375725 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42908595_6e16_4688_b583_504934328eb7.slice/crio-58fb2e1789b5af19ddc681e2f91b935f88645f5e913e00f365d2a65a18aff45c WatchSource:0}: Error finding container 58fb2e1789b5af19ddc681e2f91b935f88645f5e913e00f365d2a65a18aff45c: Status 404 returned error can't find the container with id 58fb2e1789b5af19ddc681e2f91b935f88645f5e913e00f365d2a65a18aff45c Jul 21 06:18:56.415742 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.415718 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kwzdp"] Jul 21 06:18:56.418434 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:18:56.418412 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ee4a567_fe9e_4346_ac51_1d78dede57d1.slice/crio-2df08834331e2a9f3c81cff917a1582fe4d7e3253c06108a31fa78cd46de4d79 WatchSource:0}: Error finding container 2df08834331e2a9f3c81cff917a1582fe4d7e3253c06108a31fa78cd46de4d79: Status 404 returned error can't find the container with id 2df08834331e2a9f3c81cff917a1582fe4d7e3253c06108a31fa78cd46de4d79 Jul 21 06:18:56.912464 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.912298 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l2867" event={"ID":"42908595-6e16-4688-b583-504934328eb7","Type":"ContainerStarted","Data":"58fb2e1789b5af19ddc681e2f91b935f88645f5e913e00f365d2a65a18aff45c"} Jul 21 06:18:56.914116 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.914087 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kwzdp" event={"ID":"2ee4a567-fe9e-4346-ac51-1d78dede57d1","Type":"ContainerStarted","Data":"2df08834331e2a9f3c81cff917a1582fe4d7e3253c06108a31fa78cd46de4d79"} Jul 21 06:18:56.915887 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:56.915865 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5sldl" event={"ID":"a69c086c-f31f-416b-a386-0a3e3e458acb","Type":"ContainerStarted","Data":"b638c9529b65de59315c638b67993118d518367477093f25b4673677dab073cb"} Jul 21 06:18:57.791961 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:57.791871 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:57.798359 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:57.798330 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8b9dd21a-f6c5-478c-8dc5-024a0fb145f2-metrics-tls\") pod \"dns-default-2gh49\" (UID: \"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2\") " pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:57.921541 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:57.921508 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l2867" event={"ID":"42908595-6e16-4688-b583-504934328eb7","Type":"ContainerStarted","Data":"caaccfc3a6e456b23d29b98944633142f84cb5e296c23b129d6316f3bf117e9f"} Jul 21 06:18:57.921722 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:57.921595 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l2867" event={"ID":"42908595-6e16-4688-b583-504934328eb7","Type":"ContainerStarted","Data":"2303781a978d1f0669585709c0a16210dbea7936336ee9a06270435b23b1dabc"} Jul 21 06:18:57.939854 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:57.939797 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l2867" podStartSLOduration=65.836821461 podStartE2EDuration="1m6.939775569s" totalImagesPullingTime="1.102954108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:18:57.938846879 +0000 UTC m=+67.884485537" watchObservedRunningTime="2026-07-21 06:18:57.939775569 +0000 UTC m=+67.885414228" Jul 21 06:18:58.015150 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.015119 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5sp2g\"" Jul 21 06:18:58.022658 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.022623 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2gh49" Jul 21 06:18:58.189806 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.189708 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2gh49"] Jul 21 06:18:58.192063 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:18:58.192032 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b9dd21a_f6c5_478c_8dc5_024a0fb145f2.slice/crio-3f7cfaa4e38ddaff337785ee1d64356c8947974b0ab913b89e74a1bcf19bd0d1 WatchSource:0}: Error finding container 3f7cfaa4e38ddaff337785ee1d64356c8947974b0ab913b89e74a1bcf19bd0d1: Status 404 returned error can't find the container with id 3f7cfaa4e38ddaff337785ee1d64356c8947974b0ab913b89e74a1bcf19bd0d1 Jul 21 06:18:58.394252 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.394002 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-pccmp"] Jul 21 06:18:58.422202 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.422175 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd"] Jul 21 06:18:58.422202 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.422199 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.425807 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.425786 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mr5cm\"" Jul 21 06:18:58.426037 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.426021 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 21 06:18:58.426969 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.426496 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 06:18:58.426969 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.426510 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 06:18:58.426969 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.426785 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 21 06:18:58.431361 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.431156 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 21 06:18:58.439921 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.438245 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86499bd4f9-dxnjb"] Jul 21 06:18:58.439921 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.438353 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" Jul 21 06:18:58.442007 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.441987 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cpntl\"" Jul 21 06:18:58.443085 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.442613 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 06:18:58.443085 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.442624 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 06:18:58.443085 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.442947 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 21 06:18:58.443085 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.442947 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 21 06:18:58.460126 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.460102 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-pccmp"] Jul 21 06:18:58.460263 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.460143 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd"] Jul 21 06:18:58.460263 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.460168 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86499bd4f9-dxnjb"] Jul 21 06:18:58.460263 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.460191 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86499bd4f9-dxnjb"] Jul 21 06:18:58.460263 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.460236 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-554659c859-lmjbc"] Jul 21 06:18:58.461138 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.461110 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86499bd4f9-dxnjb" Jul 21 06:18:58.476295 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.476262 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-554659c859-lmjbc"] Jul 21 06:18:58.476413 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.476331 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.479637 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.479562 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 21 06:18:58.479801 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.479785 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j7h2x\"" Jul 21 06:18:58.480407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.480346 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 21 06:18:58.480569 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.480556 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 21 06:18:58.483764 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.482579 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86499bd4f9-dxnjb" Jul 21 06:18:58.491735 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.491712 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 21 06:18:58.497451 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.497345 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e090c63e-385e-432f-a56f-b3601847ad54-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-dggkd\" (UID: \"e090c63e-385e-432f-a56f-b3601847ad54\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" Jul 21 06:18:58.497451 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.497381 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc976a1b-fc68-43d0-ad64-2b6407be665c-service-ca-bundle\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.497451 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.497408 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc976a1b-fc68-43d0-ad64-2b6407be665c-tmp\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.497451 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.497425 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/dc976a1b-fc68-43d0-ad64-2b6407be665c-snapshots\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.497705 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.497493 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e090c63e-385e-432f-a56f-b3601847ad54-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-dggkd\" (UID: \"e090c63e-385e-432f-a56f-b3601847ad54\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" Jul 21 06:18:58.497705 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.497538 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc976a1b-fc68-43d0-ad64-2b6407be665c-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.497705 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.497563 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dc976a1b-fc68-43d0-ad64-2b6407be665c-serving-cert\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.497705 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.497587 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8sxc\" (UniqueName: \"kubernetes.io/projected/dc976a1b-fc68-43d0-ad64-2b6407be665c-kube-api-access-k8sxc\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.497705 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.497616 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd5qr\" (UniqueName: \"kubernetes.io/projected/e090c63e-385e-432f-a56f-b3601847ad54-kube-api-access-bd5qr\") pod \"cluster-monitoring-operator-76cf8966f8-dggkd\" (UID: \"e090c63e-385e-432f-a56f-b3601847ad54\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" Jul 21 06:18:58.598668 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598632 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e090c63e-385e-432f-a56f-b3601847ad54-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-dggkd\" (UID: \"e090c63e-385e-432f-a56f-b3601847ad54\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" Jul 21 06:18:58.598668 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598680 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc976a1b-fc68-43d0-ad64-2b6407be665c-service-ca-bundle\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.598917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598709 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc976a1b-fc68-43d0-ad64-2b6407be665c-tmp\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.598917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598729 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/dc976a1b-fc68-43d0-ad64-2b6407be665c-snapshots\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.598917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598753 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-ca-trust-extracted\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.598917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598783 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-trusted-ca\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.598917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598805 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e090c63e-385e-432f-a56f-b3601847ad54-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-dggkd\" (UID: \"e090c63e-385e-432f-a56f-b3601847ad54\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" Jul 21 06:18:58.598917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598825 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctzlr\" (UniqueName: \"kubernetes.io/projected/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-kube-api-access-ctzlr\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.598917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598854 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc976a1b-fc68-43d0-ad64-2b6407be665c-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.598917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598874 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dc976a1b-fc68-43d0-ad64-2b6407be665c-serving-cert\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.598917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598889 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8sxc\" (UniqueName: \"kubernetes.io/projected/dc976a1b-fc68-43d0-ad64-2b6407be665c-kube-api-access-k8sxc\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.598917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598907 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-image-registry-private-configuration\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.599938 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598925 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-installation-pull-secrets\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.599938 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598945 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd5qr\" (UniqueName: \"kubernetes.io/projected/e090c63e-385e-432f-a56f-b3601847ad54-kube-api-access-bd5qr\") pod \"cluster-monitoring-operator-76cf8966f8-dggkd\" (UID: \"e090c63e-385e-432f-a56f-b3601847ad54\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" Jul 21 06:18:58.599938 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598962 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-bound-sa-token\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.599938 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598980 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-registry-tls\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.599938 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.598995 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-registry-certificates\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.599938 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.599751 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc976a1b-fc68-43d0-ad64-2b6407be665c-tmp\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.599938 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.599871 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/dc976a1b-fc68-43d0-ad64-2b6407be665c-snapshots\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.599938 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.599927 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e090c63e-385e-432f-a56f-b3601847ad54-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-dggkd\" (UID: \"e090c63e-385e-432f-a56f-b3601847ad54\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" Jul 21 06:18:58.600432 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.600199 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc976a1b-fc68-43d0-ad64-2b6407be665c-service-ca-bundle\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.600432 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.600367 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc976a1b-fc68-43d0-ad64-2b6407be665c-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.602276 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.602235 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e090c63e-385e-432f-a56f-b3601847ad54-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-dggkd\" (UID: \"e090c63e-385e-432f-a56f-b3601847ad54\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" Jul 21 06:18:58.602766 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.602747 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dc976a1b-fc68-43d0-ad64-2b6407be665c-serving-cert\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.614340 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.613949 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd5qr\" (UniqueName: \"kubernetes.io/projected/e090c63e-385e-432f-a56f-b3601847ad54-kube-api-access-bd5qr\") pod \"cluster-monitoring-operator-76cf8966f8-dggkd\" (UID: \"e090c63e-385e-432f-a56f-b3601847ad54\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" Jul 21 06:18:58.614340 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.614180 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8sxc\" (UniqueName: \"kubernetes.io/projected/dc976a1b-fc68-43d0-ad64-2b6407be665c-kube-api-access-k8sxc\") pod \"insights-operator-555d9d7885-pccmp\" (UID: \"dc976a1b-fc68-43d0-ad64-2b6407be665c\") " pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.699613 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.699543 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-image-registry-private-configuration\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.699613 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.699591 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-installation-pull-secrets\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.699823 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.699742 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-bound-sa-token\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.699823 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.699790 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-registry-tls\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.699823 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.699815 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-registry-certificates\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.699968 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.699870 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-ca-trust-extracted\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.699968 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.699905 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-trusted-ca\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.699968 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.699930 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctzlr\" (UniqueName: \"kubernetes.io/projected/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-kube-api-access-ctzlr\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.701051 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.701025 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-ca-trust-extracted\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.701260 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.701237 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-registry-certificates\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.701884 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.701862 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-trusted-ca\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.702730 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.702690 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-registry-tls\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.703318 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.703296 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-installation-pull-secrets\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.704820 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.704798 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-image-registry-private-configuration\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.713050 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.713031 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-bound-sa-token\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.713315 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.713288 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctzlr\" (UniqueName: \"kubernetes.io/projected/1db48e88-ac22-4412-bfd7-cd4fde7a37e8-kube-api-access-ctzlr\") pod \"image-registry-554659c859-lmjbc\" (UID: \"1db48e88-ac22-4412-bfd7-cd4fde7a37e8\") " pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.735537 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.735334 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-pccmp" Jul 21 06:18:58.750343 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.750321 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" Jul 21 06:18:58.789197 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.789032 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:18:58.926312 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.926277 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2gh49" event={"ID":"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2","Type":"ContainerStarted","Data":"3f7cfaa4e38ddaff337785ee1d64356c8947974b0ab913b89e74a1bcf19bd0d1"} Jul 21 06:18:58.926312 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.926301 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86499bd4f9-dxnjb" Jul 21 06:18:58.964483 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.964377 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86499bd4f9-dxnjb"] Jul 21 06:18:58.967292 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:18:58.967263 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-86499bd4f9-dxnjb"] Jul 21 06:19:01.282997 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.282888 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-pccmp"] Jul 21 06:19:01.287461 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:19:01.287408 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc976a1b_fc68_43d0_ad64_2b6407be665c.slice/crio-197245432df1bf373d24112300e052e84b56025f263e8338e5d7fac369e89216 WatchSource:0}: Error finding container 197245432df1bf373d24112300e052e84b56025f263e8338e5d7fac369e89216: Status 404 returned error can't find the container with id 197245432df1bf373d24112300e052e84b56025f263e8338e5d7fac369e89216 Jul 21 06:19:01.300399 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.300343 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-554659c859-lmjbc"] Jul 21 06:19:01.302493 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.302429 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd"] Jul 21 06:19:01.305184 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:19:01.305147 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db48e88_ac22_4412_bfd7_cd4fde7a37e8.slice/crio-d5f8c7cdc2a79cca566b67a480e4b8a03123608243f6fc0446c0b08a02cccc8c WatchSource:0}: Error finding container d5f8c7cdc2a79cca566b67a480e4b8a03123608243f6fc0446c0b08a02cccc8c: Status 404 returned error can't find the container with id d5f8c7cdc2a79cca566b67a480e4b8a03123608243f6fc0446c0b08a02cccc8c Jul 21 06:19:01.306400 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:19:01.306381 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode090c63e_385e_432f_a56f_b3601847ad54.slice/crio-37646e1dc1649935415ed6e427ac5089cda91c5a55ea57b514ceedd6148f04a3 WatchSource:0}: Error finding container 37646e1dc1649935415ed6e427ac5089cda91c5a55ea57b514ceedd6148f04a3: Status 404 returned error can't find the container with id 37646e1dc1649935415ed6e427ac5089cda91c5a55ea57b514ceedd6148f04a3 Jul 21 06:19:01.939099 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.939061 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" event={"ID":"e090c63e-385e-432f-a56f-b3601847ad54","Type":"ContainerStarted","Data":"37646e1dc1649935415ed6e427ac5089cda91c5a55ea57b514ceedd6148f04a3"} Jul 21 06:19:01.941450 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.941423 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kwzdp" event={"ID":"2ee4a567-fe9e-4346-ac51-1d78dede57d1","Type":"ContainerStarted","Data":"6f78b117a78a4a6d26a8d99594113dbded7b336cfb672ac8e57f1c6124b20207"} Jul 21 06:19:01.943102 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.943080 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-554659c859-lmjbc" event={"ID":"1db48e88-ac22-4412-bfd7-cd4fde7a37e8","Type":"ContainerStarted","Data":"b65b29d111ab898117934d249abd8e982f371ebc27dd627ffe94a9125630fc8d"} Jul 21 06:19:01.943187 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.943123 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-554659c859-lmjbc" event={"ID":"1db48e88-ac22-4412-bfd7-cd4fde7a37e8","Type":"ContainerStarted","Data":"d5f8c7cdc2a79cca566b67a480e4b8a03123608243f6fc0446c0b08a02cccc8c"} Jul 21 06:19:01.943604 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.943587 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:19:01.945322 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.945295 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5sldl" event={"ID":"a69c086c-f31f-416b-a386-0a3e3e458acb","Type":"ContainerStarted","Data":"53e1d293ae78a6d557f12f19fb7d22ffc10ee9f0f4d5d1a25cff11f95352841d"} Jul 21 06:19:01.945644 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.945627 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:19:01.946754 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.946735 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-pccmp" event={"ID":"dc976a1b-fc68-43d0-ad64-2b6407be665c","Type":"ContainerStarted","Data":"197245432df1bf373d24112300e052e84b56025f263e8338e5d7fac369e89216"} Jul 21 06:19:01.957655 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.957072 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kwzdp" podStartSLOduration=66.239992333 podStartE2EDuration="1m10.957059963s" totalImagesPullingTime="4.71706763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 06:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:01.956855497 +0000 UTC m=+71.902494155" watchObservedRunningTime="2026-07-21 06:19:01.957059963 +0000 UTC m=+71.902698619" Jul 21 06:19:01.977047 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.976993 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-554659c859-lmjbc" podStartSLOduration=3.97697774 podStartE2EDuration="3.97697774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:18:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:01.976538453 +0000 UTC m=+71.922177111" watchObservedRunningTime="2026-07-21 06:19:01.97697774 +0000 UTC m=+71.922616396" Jul 21 06:19:01.992292 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:01.991436 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5sldl" podStartSLOduration=66.231212406 podStartE2EDuration="1m10.991424123s" totalImagesPullingTime="4.760211717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:01.990847081 +0000 UTC m=+71.936485737" watchObservedRunningTime="2026-07-21 06:19:01.991424123 +0000 UTC m=+71.937062780" Jul 21 06:19:02.955686 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:02.955246 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2gh49" event={"ID":"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2","Type":"ContainerStarted","Data":"48a458572d5df09875d577e594433a34ae49e3ee2c3b38558b2e2c5fc6f31fb8"} Jul 21 06:19:02.955686 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:02.955365 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2gh49" event={"ID":"8b9dd21a-f6c5-478c-8dc5-024a0fb145f2","Type":"ContainerStarted","Data":"2f881b7d62459e2ea5e134347716a28aea7c96d416f9679e54be4be3313845d0"} Jul 21 06:19:02.956179 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:02.955722 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2gh49" Jul 21 06:19:02.972881 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:02.972824 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2gh49" podStartSLOduration=33.630498182 podStartE2EDuration="37.972811259s" totalImagesPullingTime="4.342313077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:02.972418175 +0000 UTC m=+72.918056833" watchObservedRunningTime="2026-07-21 06:19:02.972811259 +0000 UTC m=+72.918449916" Jul 21 06:19:03.959764 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:03.959676 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" event={"ID":"e090c63e-385e-432f-a56f-b3601847ad54","Type":"ContainerStarted","Data":"a789e0e13eeba7d4b6a47697e207ab418dc419349b1bdd8594aeabb4f7958ac5"} Jul 21 06:19:03.961366 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:03.961333 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-pccmp" event={"ID":"dc976a1b-fc68-43d0-ad64-2b6407be665c","Type":"ContainerStarted","Data":"8ceb5f88c77d2e390509c00900593e719d8736d77d733bbc76f644edd3e33b17"} Jul 21 06:19:03.978697 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:03.978650 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dggkd" podStartSLOduration=3.610769329 podStartE2EDuration="5.978631566s" totalImagesPullingTime="2.367862237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:18:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:03.977442535 +0000 UTC m=+73.923081193" watchObservedRunningTime="2026-07-21 06:19:03.978631566 +0000 UTC m=+73.924270224" Jul 21 06:19:03.999266 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:03.999204 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-pccmp" podStartSLOduration=3.609292933 podStartE2EDuration="5.999188067s" totalImagesPullingTime="2.389895134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:18:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:03.99809604 +0000 UTC m=+73.943734698" watchObservedRunningTime="2026-07-21 06:19:03.999188067 +0000 UTC m=+73.944826726" Jul 21 06:19:04.163757 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.163721 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr"] Jul 21 06:19:04.165552 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.165535 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr" Jul 21 06:19:04.168386 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.168362 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 21 06:19:04.168496 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.168366 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6vm4q\"" Jul 21 06:19:04.174540 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.174518 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr"] Jul 21 06:19:04.241300 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.241230 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c55958ac-8a82-471d-8af9-90012d8aee16-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-zrffr\" (UID: \"c55958ac-8a82-471d-8af9-90012d8aee16\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr" Jul 21 06:19:04.342511 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.342479 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c55958ac-8a82-471d-8af9-90012d8aee16-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-zrffr\" (UID: \"c55958ac-8a82-471d-8af9-90012d8aee16\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr" Jul 21 06:19:04.342674 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:04.342582 2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 21 06:19:04.342674 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:04.342636 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c55958ac-8a82-471d-8af9-90012d8aee16-tls-certificates podName:c55958ac-8a82-471d-8af9-90012d8aee16 nodeName:}" failed. No retries permitted until 2026-07-21 06:19:04.842620359 +0000 UTC m=+74.788258993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/c55958ac-8a82-471d-8af9-90012d8aee16-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-zrffr" (UID: "c55958ac-8a82-471d-8af9-90012d8aee16") : secret "prometheus-operator-admission-webhook-tls" not found Jul 21 06:19:04.511277 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.511191 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zzv9d"] Jul 21 06:19:04.513177 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.513155 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.516038 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.516019 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 06:19:04.516038 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.516025 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 06:19:04.516238 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.516032 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7g9pq\"" Jul 21 06:19:04.525848 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.525831 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zzv9d"] Jul 21 06:19:04.543713 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.543683 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/05fea74d-5017-4bf7-b70b-70919bb2c4a6-crio-socket\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.543818 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.543751 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/05fea74d-5017-4bf7-b70b-70919bb2c4a6-data-volume\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.543860 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.543814 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.543901 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.543876 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hvfs\" (UniqueName: \"kubernetes.io/projected/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-api-access-5hvfs\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.543932 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.543904 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.644512 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.644484 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.644653 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.644527 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/05fea74d-5017-4bf7-b70b-70919bb2c4a6-crio-socket\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.644653 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.644567 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/05fea74d-5017-4bf7-b70b-70919bb2c4a6-data-volume\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.644653 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.644594 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.644653 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:04.644600 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 06:19:04.644832 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:04.644674 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm podName:05fea74d-5017-4bf7-b70b-70919bb2c4a6 nodeName:}" failed. No retries permitted until 2026-07-21 06:19:05.144654026 +0000 UTC m=+75.090292676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzv9d" (UID: "05fea74d-5017-4bf7-b70b-70919bb2c4a6") : configmap "kube-rbac-proxy" not found Jul 21 06:19:04.644832 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.644680 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/05fea74d-5017-4bf7-b70b-70919bb2c4a6-crio-socket\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.644832 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:04.644712 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 06:19:04.644832 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.644720 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hvfs\" (UniqueName: \"kubernetes.io/projected/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-api-access-5hvfs\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.644832 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:04.644761 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls podName:05fea74d-5017-4bf7-b70b-70919bb2c4a6 nodeName:}" failed. No retries permitted until 2026-07-21 06:19:05.144745923 +0000 UTC m=+75.090384572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzv9d" (UID: "05fea74d-5017-4bf7-b70b-70919bb2c4a6") : secret "insights-runtime-extractor-tls" not found Jul 21 06:19:04.644996 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.644918 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/05fea74d-5017-4bf7-b70b-70919bb2c4a6-data-volume\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.653036 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.653010 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hvfs\" (UniqueName: \"kubernetes.io/projected/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-api-access-5hvfs\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:04.846354 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.846327 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c55958ac-8a82-471d-8af9-90012d8aee16-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-zrffr\" (UID: \"c55958ac-8a82-471d-8af9-90012d8aee16\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr" Jul 21 06:19:04.848859 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:04.848837 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c55958ac-8a82-471d-8af9-90012d8aee16-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-zrffr\" (UID: \"c55958ac-8a82-471d-8af9-90012d8aee16\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr" Jul 21 06:19:05.075583 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:05.075552 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr" Jul 21 06:19:05.117099 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:05.117006 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2gh49_8b9dd21a-f6c5-478c-8dc5-024a0fb145f2/dns/0.log" Jul 21 06:19:05.148538 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:05.148506 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:05.148703 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:05.148554 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:05.148703 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:05.148632 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 06:19:05.148703 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:05.148661 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 06:19:05.148703 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:05.148695 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm podName:05fea74d-5017-4bf7-b70b-70919bb2c4a6 nodeName:}" failed. No retries permitted until 2026-07-21 06:19:06.148679633 +0000 UTC m=+76.094318268 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzv9d" (UID: "05fea74d-5017-4bf7-b70b-70919bb2c4a6") : configmap "kube-rbac-proxy" not found Jul 21 06:19:05.148890 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:05.148733 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls podName:05fea74d-5017-4bf7-b70b-70919bb2c4a6 nodeName:}" failed. No retries permitted until 2026-07-21 06:19:06.148718248 +0000 UTC m=+76.094356900 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzv9d" (UID: "05fea74d-5017-4bf7-b70b-70919bb2c4a6") : secret "insights-runtime-extractor-tls" not found Jul 21 06:19:05.195748 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:05.195725 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr"] Jul 21 06:19:05.197927 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:19:05.197900 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc55958ac_8a82_471d_8af9_90012d8aee16.slice/crio-bc2bfeb5d1d98152b8f5f858c06ef9153abfb1a6c661385fb598947c749bf697 WatchSource:0}: Error finding container bc2bfeb5d1d98152b8f5f858c06ef9153abfb1a6c661385fb598947c749bf697: Status 404 returned error can't find the container with id bc2bfeb5d1d98152b8f5f858c06ef9153abfb1a6c661385fb598947c749bf697 Jul 21 06:19:05.294321 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:05.294292 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2gh49_8b9dd21a-f6c5-478c-8dc5-024a0fb145f2/kube-rbac-proxy/0.log" Jul 21 06:19:05.969099 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:05.969061 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr" event={"ID":"c55958ac-8a82-471d-8af9-90012d8aee16","Type":"ContainerStarted","Data":"bc2bfeb5d1d98152b8f5f858c06ef9153abfb1a6c661385fb598947c749bf697"} Jul 21 06:19:06.155099 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:06.155067 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:06.155520 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:06.155153 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:06.155520 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:06.155191 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 06:19:06.155520 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:06.155290 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm podName:05fea74d-5017-4bf7-b70b-70919bb2c4a6 nodeName:}" failed. No retries permitted until 2026-07-21 06:19:08.155270445 +0000 UTC m=+78.100909093 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzv9d" (UID: "05fea74d-5017-4bf7-b70b-70919bb2c4a6") : configmap "kube-rbac-proxy" not found Jul 21 06:19:06.155520 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:06.155397 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 06:19:06.155520 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:06.155456 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls podName:05fea74d-5017-4bf7-b70b-70919bb2c4a6 nodeName:}" failed. No retries permitted until 2026-07-21 06:19:08.155440278 +0000 UTC m=+78.101078929 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzv9d" (UID: "05fea74d-5017-4bf7-b70b-70919bb2c4a6") : secret "insights-runtime-extractor-tls" not found Jul 21 06:19:06.497341 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:06.497314 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ks8rh_512070fa-797a-4c81-92f3-0510a4c7cfbb/dns-node-resolver/0.log" Jul 21 06:19:06.894403 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:06.894374 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-554659c859-lmjbc_1db48e88-ac22-4412-bfd7-cd4fde7a37e8/registry/0.log" Jul 21 06:19:06.973207 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:06.973180 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr" event={"ID":"c55958ac-8a82-471d-8af9-90012d8aee16","Type":"ContainerStarted","Data":"197cc0adf9bff940c820a78a24e9fa7459bb07c670585ee81f950f4ee57d88cf"} Jul 21 06:19:06.973386 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:06.973373 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr" Jul 21 06:19:06.977917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:06.977894 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr" Jul 21 06:19:06.989840 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:06.989799 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-zrffr" podStartSLOduration=1.8168521119999999 podStartE2EDuration="2.98978483s" totalImagesPullingTime="1.172932718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:06.989526939 +0000 UTC m=+76.935165596" watchObservedRunningTime="2026-07-21 06:19:06.98978483 +0000 UTC m=+76.935423514" Jul 21 06:19:07.294170 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:07.294096 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6dtks_7b9eedde-6415-425e-9223-d74b2a3f74cb/node-ca/0.log" Jul 21 06:19:08.171007 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:08.170972 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:08.171203 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:08.171027 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:08.171203 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:08.171112 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 06:19:08.171203 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:08.171124 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 06:19:08.171203 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:08.171178 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls podName:05fea74d-5017-4bf7-b70b-70919bb2c4a6 nodeName:}" failed. No retries permitted until 2026-07-21 06:19:12.171165517 +0000 UTC m=+82.116804152 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzv9d" (UID: "05fea74d-5017-4bf7-b70b-70919bb2c4a6") : secret "insights-runtime-extractor-tls" not found Jul 21 06:19:08.171203 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:08.171199 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm podName:05fea74d-5017-4bf7-b70b-70919bb2c4a6 nodeName:}" failed. No retries permitted until 2026-07-21 06:19:12.171181673 +0000 UTC m=+82.116820328 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzv9d" (UID: "05fea74d-5017-4bf7-b70b-70919bb2c4a6") : configmap "kube-rbac-proxy" not found Jul 21 06:19:08.494386 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:08.494310 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w6zct_c253498b-ea18-4b4d-b21d-116381a32d7f/serve-healthcheck-canary/0.log" Jul 21 06:19:12.205242 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.205107 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:12.205242 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.205156 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:12.205658 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:12.205249 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 06:19:12.205658 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:12.205312 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 06:19:12.205658 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:12.205317 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm podName:05fea74d-5017-4bf7-b70b-70919bb2c4a6 nodeName:}" failed. No retries permitted until 2026-07-21 06:19:20.205297349 +0000 UTC m=+90.150935996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zzv9d" (UID: "05fea74d-5017-4bf7-b70b-70919bb2c4a6") : configmap "kube-rbac-proxy" not found Jul 21 06:19:12.205658 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:19:12.205373 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls podName:05fea74d-5017-4bf7-b70b-70919bb2c4a6 nodeName:}" failed. No retries permitted until 2026-07-21 06:19:20.205358873 +0000 UTC m=+90.150997508 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zzv9d" (UID: "05fea74d-5017-4bf7-b70b-70919bb2c4a6") : secret "insights-runtime-extractor-tls" not found Jul 21 06:19:12.472875 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.472802 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f45bbdb99-rphzc"] Jul 21 06:19:12.476245 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.476208 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.480922 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.480898 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zcncg\"" Jul 21 06:19:12.481352 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.480898 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 21 06:19:12.481352 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.480927 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 21 06:19:12.481352 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.480931 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 21 06:19:12.481531 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.480984 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 21 06:19:12.481531 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.480989 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 21 06:19:12.481531 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.481015 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 21 06:19:12.481531 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.481030 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 21 06:19:12.486415 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.486388 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 21 06:19:12.486915 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.486896 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f45bbdb99-rphzc"] Jul 21 06:19:12.609515 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.609482 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-oauth-config\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.609515 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.609517 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-config\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.609779 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.609568 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-oauth-serving-cert\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.609779 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.609585 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-service-ca\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.609779 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.609632 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8jm4\" (UniqueName: \"kubernetes.io/projected/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-kube-api-access-h8jm4\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.609779 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.609705 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-serving-cert\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.609779 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.609774 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-trusted-ca-bundle\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.710874 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.710822 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-oauth-serving-cert\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.710874 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.710875 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-service-ca\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.711114 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.710902 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8jm4\" (UniqueName: \"kubernetes.io/projected/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-kube-api-access-h8jm4\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.711114 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.710924 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-serving-cert\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.711114 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.710957 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-trusted-ca-bundle\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.711114 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.710978 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-oauth-config\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.711114 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.710995 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-config\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.711691 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.711662 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-oauth-serving-cert\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.711819 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.711797 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-config\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.711949 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.711935 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-trusted-ca-bundle\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.711991 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.711948 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-service-ca\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.713513 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.713493 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-serving-cert\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.713513 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.713503 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-oauth-config\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.721687 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.719896 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8jm4\" (UniqueName: \"kubernetes.io/projected/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-kube-api-access-h8jm4\") pod \"console-f45bbdb99-rphzc\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.787183 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.787093 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:12.915622 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.915593 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f45bbdb99-rphzc"] Jul 21 06:19:12.918661 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:19:12.918636 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d9b87fc_21c4_4340_b8c3_8895cb620ad1.slice/crio-16c3c0fe8f8c37d697b872d2acc7c38953ba1d16345b4dc987113de670e4c5ec WatchSource:0}: Error finding container 16c3c0fe8f8c37d697b872d2acc7c38953ba1d16345b4dc987113de670e4c5ec: Status 404 returned error can't find the container with id 16c3c0fe8f8c37d697b872d2acc7c38953ba1d16345b4dc987113de670e4c5ec Jul 21 06:19:12.994770 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:12.994732 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f45bbdb99-rphzc" event={"ID":"2d9b87fc-21c4-4340-b8c3-8895cb620ad1","Type":"ContainerStarted","Data":"16c3c0fe8f8c37d697b872d2acc7c38953ba1d16345b4dc987113de670e4c5ec"} Jul 21 06:19:13.966945 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:13.966913 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2gh49" Jul 21 06:19:16.006813 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:16.006732 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f45bbdb99-rphzc" event={"ID":"2d9b87fc-21c4-4340-b8c3-8895cb620ad1","Type":"ContainerStarted","Data":"dcec55f15c09ed7e83e27e4f4b0fc5d19b9bbbbcb4b7349ccbd461594fb24b03"} Jul 21 06:19:16.026831 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:16.026780 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f45bbdb99-rphzc" podStartSLOduration=1.276903579 podStartE2EDuration="4.0267638s" totalImagesPullingTime="2.749860221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:16.024822831 +0000 UTC m=+85.970461488" watchObservedRunningTime="2026-07-21 06:19:16.0267638 +0000 UTC m=+85.972402457" Jul 21 06:19:19.541261 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.541208 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7"] Jul 21 06:19:19.544548 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.544532 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.547459 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.547436 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-jxvhp\"" Jul 21 06:19:19.549111 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.549092 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 06:19:19.549265 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.549120 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 21 06:19:19.549265 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.549120 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 21 06:19:19.553755 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.553734 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7"] Jul 21 06:19:19.556596 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.556576 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-7f792"] Jul 21 06:19:19.559898 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.559878 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.563088 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.563070 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 21 06:19:19.563201 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.563132 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-lzgrl\"" Jul 21 06:19:19.563201 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.563158 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 21 06:19:19.563437 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.563415 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 21 06:19:19.569826 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.569789 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gqqrw"] Jul 21 06:19:19.572904 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.572875 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-7f792"] Jul 21 06:19:19.573008 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.572991 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.576085 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.575997 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 06:19:19.576085 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.576012 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hsg87\"" Jul 21 06:19:19.576085 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.576004 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 06:19:19.576325 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.576017 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 06:19:19.670036 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670005 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-accelerators-collector-config\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.670234 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670042 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hrvd\" (UniqueName: \"kubernetes.io/projected/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-kube-api-access-8hrvd\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.670234 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670069 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-wtmp\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.670234 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670095 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-tls\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.670234 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670160 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3bc626c-3b7d-4689-baa0-4c22c56360d3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.670234 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670195 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4eb512c1-6716-43c5-a14b-c88e0a2eaf14-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-5v4f7\" (UID: \"4eb512c1-6716-43c5-a14b-c88e0a2eaf14\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.670492 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670279 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c3bc626c-3b7d-4689-baa0-4c22c56360d3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.670492 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670343 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3bc626c-3b7d-4689-baa0-4c22c56360d3-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.670492 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670377 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.670492 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670415 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-textfile\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.670492 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670466 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3bc626c-3b7d-4689-baa0-4c22c56360d3-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.670741 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670491 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwwt2\" (UniqueName: \"kubernetes.io/projected/4eb512c1-6716-43c5-a14b-c88e0a2eaf14-kube-api-access-vwwt2\") pod \"openshift-state-metrics-79f6958cdf-5v4f7\" (UID: \"4eb512c1-6716-43c5-a14b-c88e0a2eaf14\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.670741 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670522 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4eb512c1-6716-43c5-a14b-c88e0a2eaf14-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-5v4f7\" (UID: \"4eb512c1-6716-43c5-a14b-c88e0a2eaf14\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.670741 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670548 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbvbm\" (UniqueName: \"kubernetes.io/projected/c3bc626c-3b7d-4689-baa0-4c22c56360d3-kube-api-access-jbvbm\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.670741 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670573 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-metrics-client-ca\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.670741 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670595 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4eb512c1-6716-43c5-a14b-c88e0a2eaf14-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-5v4f7\" (UID: \"4eb512c1-6716-43c5-a14b-c88e0a2eaf14\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.670741 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670659 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c3bc626c-3b7d-4689-baa0-4c22c56360d3-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.670741 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670676 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-sys\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.670741 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.670696 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-root\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.772036 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.771999 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrvd\" (UniqueName: \"kubernetes.io/projected/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-kube-api-access-8hrvd\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.772036 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772038 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-wtmp\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.772305 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772065 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-tls\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.772305 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772091 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3bc626c-3b7d-4689-baa0-4c22c56360d3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.772305 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772114 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4eb512c1-6716-43c5-a14b-c88e0a2eaf14-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-5v4f7\" (UID: \"4eb512c1-6716-43c5-a14b-c88e0a2eaf14\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.772305 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772158 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c3bc626c-3b7d-4689-baa0-4c22c56360d3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.772305 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772183 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3bc626c-3b7d-4689-baa0-4c22c56360d3-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.772305 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772211 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.772305 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772258 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-wtmp\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.772305 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772273 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-textfile\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.772724 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772349 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3bc626c-3b7d-4689-baa0-4c22c56360d3-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.772724 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772379 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwwt2\" (UniqueName: \"kubernetes.io/projected/4eb512c1-6716-43c5-a14b-c88e0a2eaf14-kube-api-access-vwwt2\") pod \"openshift-state-metrics-79f6958cdf-5v4f7\" (UID: \"4eb512c1-6716-43c5-a14b-c88e0a2eaf14\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.772724 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772599 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-textfile\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.772724 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772638 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4eb512c1-6716-43c5-a14b-c88e0a2eaf14-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-5v4f7\" (UID: \"4eb512c1-6716-43c5-a14b-c88e0a2eaf14\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.772724 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772670 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbvbm\" (UniqueName: \"kubernetes.io/projected/c3bc626c-3b7d-4689-baa0-4c22c56360d3-kube-api-access-jbvbm\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.772724 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772692 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-metrics-client-ca\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.772724 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772720 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4eb512c1-6716-43c5-a14b-c88e0a2eaf14-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-5v4f7\" (UID: \"4eb512c1-6716-43c5-a14b-c88e0a2eaf14\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.773295 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772758 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c3bc626c-3b7d-4689-baa0-4c22c56360d3-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.773295 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772794 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-sys\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.773295 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772823 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-root\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.773295 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.772882 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-accelerators-collector-config\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.773494 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.773416 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c3bc626c-3b7d-4689-baa0-4c22c56360d3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.773752 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.773715 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-accelerators-collector-config\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.773849 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.773827 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3bc626c-3b7d-4689-baa0-4c22c56360d3-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.774097 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.774071 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c3bc626c-3b7d-4689-baa0-4c22c56360d3-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.774201 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.774119 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4eb512c1-6716-43c5-a14b-c88e0a2eaf14-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-5v4f7\" (UID: \"4eb512c1-6716-43c5-a14b-c88e0a2eaf14\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.774588 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.774318 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-metrics-client-ca\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.774588 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.774334 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-sys\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.774588 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.774423 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-root\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.775816 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.775776 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3bc626c-3b7d-4689-baa0-4c22c56360d3-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.776472 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.776412 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.776632 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.776580 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-node-exporter-tls\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.776732 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.776620 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4eb512c1-6716-43c5-a14b-c88e0a2eaf14-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-5v4f7\" (UID: \"4eb512c1-6716-43c5-a14b-c88e0a2eaf14\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.777312 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.777290 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3bc626c-3b7d-4689-baa0-4c22c56360d3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.777417 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.777329 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4eb512c1-6716-43c5-a14b-c88e0a2eaf14-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-5v4f7\" (UID: \"4eb512c1-6716-43c5-a14b-c88e0a2eaf14\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.781302 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.781193 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hrvd\" (UniqueName: \"kubernetes.io/projected/a7ea94d1-3d5b-4f1e-a587-b09b12fedeae-kube-api-access-8hrvd\") pod \"node-exporter-gqqrw\" (UID: \"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae\") " pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.782397 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.782373 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbvbm\" (UniqueName: \"kubernetes.io/projected/c3bc626c-3b7d-4689-baa0-4c22c56360d3-kube-api-access-jbvbm\") pod \"kube-state-metrics-56c5594674-7f792\" (UID: \"c3bc626c-3b7d-4689-baa0-4c22c56360d3\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.783353 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.783325 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwwt2\" (UniqueName: \"kubernetes.io/projected/4eb512c1-6716-43c5-a14b-c88e0a2eaf14-kube-api-access-vwwt2\") pod \"openshift-state-metrics-79f6958cdf-5v4f7\" (UID: \"4eb512c1-6716-43c5-a14b-c88e0a2eaf14\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.853748 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.853711 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" Jul 21 06:19:19.867899 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.867875 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" Jul 21 06:19:19.881959 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:19.881932 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gqqrw" Jul 21 06:19:19.893831 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:19:19.893794 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ea94d1_3d5b_4f1e_a587_b09b12fedeae.slice/crio-f06fbdc1f1e86b1b864ef3975c5d1ede4dda8637b26a363665963328280d1f98 WatchSource:0}: Error finding container f06fbdc1f1e86b1b864ef3975c5d1ede4dda8637b26a363665963328280d1f98: Status 404 returned error can't find the container with id f06fbdc1f1e86b1b864ef3975c5d1ede4dda8637b26a363665963328280d1f98 Jul 21 06:19:20.004590 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.004569 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7"] Jul 21 06:19:20.006896 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:19:20.006870 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb512c1_6716_43c5_a14b_c88e0a2eaf14.slice/crio-6bdf9c8f574a7cb7530f6167eacc0ee4c18e37ba6d64d5eda8a59966f1e1f632 WatchSource:0}: Error finding container 6bdf9c8f574a7cb7530f6167eacc0ee4c18e37ba6d64d5eda8a59966f1e1f632: Status 404 returned error can't find the container with id 6bdf9c8f574a7cb7530f6167eacc0ee4c18e37ba6d64d5eda8a59966f1e1f632 Jul 21 06:19:20.021716 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.018353 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-7f792"] Jul 21 06:19:20.023376 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:19:20.023280 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3bc626c_3b7d_4689_baa0_4c22c56360d3.slice/crio-2ddcbc6e33c21ed25afafa7602bb62b6ea05f9a23c077ddd9f23348ee6368ec9 WatchSource:0}: Error finding container 2ddcbc6e33c21ed25afafa7602bb62b6ea05f9a23c077ddd9f23348ee6368ec9: Status 404 returned error can't find the container with id 2ddcbc6e33c21ed25afafa7602bb62b6ea05f9a23c077ddd9f23348ee6368ec9 Jul 21 06:19:20.025283 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.025238 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" event={"ID":"4eb512c1-6716-43c5-a14b-c88e0a2eaf14","Type":"ContainerStarted","Data":"6bdf9c8f574a7cb7530f6167eacc0ee4c18e37ba6d64d5eda8a59966f1e1f632"} Jul 21 06:19:20.026896 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.026871 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gqqrw" event={"ID":"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae","Type":"ContainerStarted","Data":"f06fbdc1f1e86b1b864ef3975c5d1ede4dda8637b26a363665963328280d1f98"} Jul 21 06:19:20.277681 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.277640 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:20.277884 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.277709 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:20.278245 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.278196 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/05fea74d-5017-4bf7-b70b-70919bb2c4a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:20.280305 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.280285 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/05fea74d-5017-4bf7-b70b-70919bb2c4a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zzv9d\" (UID: \"05fea74d-5017-4bf7-b70b-70919bb2c4a6\") " pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:20.421660 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.421578 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zzv9d" Jul 21 06:19:20.611780 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.611737 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zzv9d"] Jul 21 06:19:20.615652 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:19:20.615620 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05fea74d_5017_4bf7_b70b_70919bb2c4a6.slice/crio-5c173efc1970ab6b8b80d8b3bafe3118f8e56325137b2352096c7281d59b7599 WatchSource:0}: Error finding container 5c173efc1970ab6b8b80d8b3bafe3118f8e56325137b2352096c7281d59b7599: Status 404 returned error can't find the container with id 5c173efc1970ab6b8b80d8b3bafe3118f8e56325137b2352096c7281d59b7599 Jul 21 06:19:20.658415 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.658380 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 06:19:20.662380 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.662356 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.665589 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.665562 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 21 06:19:20.665828 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.665800 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 21 06:19:20.666954 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.666332 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 21 06:19:20.666954 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.666619 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 21 06:19:20.666954 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.666625 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 21 06:19:20.666954 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.666857 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 21 06:19:20.667283 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.667180 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 21 06:19:20.667830 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.667411 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 21 06:19:20.667830 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.667604 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 21 06:19:20.668021 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.667898 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kb6mw\"" Jul 21 06:19:20.679022 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.678874 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 06:19:20.790246 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.790180 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg9wz\" (UniqueName: \"kubernetes.io/projected/7150569a-b586-4793-a185-e889dee4ee13-kube-api-access-gg9wz\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.790417 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.790258 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.790417 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.790291 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.790417 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.790317 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7150569a-b586-4793-a185-e889dee4ee13-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.790417 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.790343 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-config-volume\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.790625 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.790454 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7150569a-b586-4793-a185-e889dee4ee13-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.790625 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.790530 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-web-config\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.790625 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.790603 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7150569a-b586-4793-a185-e889dee4ee13-config-out\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.790776 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.790667 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.790776 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.790749 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.790879 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.790831 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.791393 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.790962 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7150569a-b586-4793-a185-e889dee4ee13-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.791393 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.791080 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7150569a-b586-4793-a185-e889dee4ee13-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.891814 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.891779 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.891981 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.891832 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.891981 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.891864 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.891981 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.891892 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7150569a-b586-4793-a185-e889dee4ee13-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.891981 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.891955 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7150569a-b586-4793-a185-e889dee4ee13-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.892198 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.891980 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg9wz\" (UniqueName: \"kubernetes.io/projected/7150569a-b586-4793-a185-e889dee4ee13-kube-api-access-gg9wz\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.892198 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.892006 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.892198 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.892025 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.892198 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.892058 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7150569a-b586-4793-a185-e889dee4ee13-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.892198 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.892084 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-config-volume\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.892198 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.892153 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7150569a-b586-4793-a185-e889dee4ee13-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.892198 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.892177 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-web-config\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.892198 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.892201 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7150569a-b586-4793-a185-e889dee4ee13-config-out\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.893888 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.893857 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7150569a-b586-4793-a185-e889dee4ee13-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.894852 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.894824 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7150569a-b586-4793-a185-e889dee4ee13-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.894976 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.894933 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7150569a-b586-4793-a185-e889dee4ee13-config-out\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.896689 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.896661 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7150569a-b586-4793-a185-e889dee4ee13-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.899705 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.899660 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.900203 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.900183 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.900320 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.900187 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-web-config\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.900473 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.900450 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7150569a-b586-4793-a185-e889dee4ee13-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.900606 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.900576 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-config-volume\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.900759 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.900735 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.901161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.901139 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.904392 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.904082 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg9wz\" (UniqueName: \"kubernetes.io/projected/7150569a-b586-4793-a185-e889dee4ee13-kube-api-access-gg9wz\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.906919 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.906890 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:20.981040 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:20.980954 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:19:21.032403 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:21.032362 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zzv9d" event={"ID":"05fea74d-5017-4bf7-b70b-70919bb2c4a6","Type":"ContainerStarted","Data":"5c173efc1970ab6b8b80d8b3bafe3118f8e56325137b2352096c7281d59b7599"} Jul 21 06:19:21.034495 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:21.034468 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" event={"ID":"4eb512c1-6716-43c5-a14b-c88e0a2eaf14","Type":"ContainerStarted","Data":"ac5a45a0157c3d045b6e237e0c2fafb61b7c2cbe8b2ac4ad98f7cc31a2aebfed"} Jul 21 06:19:21.034631 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:21.034532 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" event={"ID":"4eb512c1-6716-43c5-a14b-c88e0a2eaf14","Type":"ContainerStarted","Data":"2a34695575cd32ea65be845689a0e5b36e6867682278a140b465f968e52fde33"} Jul 21 06:19:21.035651 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:21.035591 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" event={"ID":"c3bc626c-3b7d-4689-baa0-4c22c56360d3","Type":"ContainerStarted","Data":"2ddcbc6e33c21ed25afafa7602bb62b6ea05f9a23c077ddd9f23348ee6368ec9"} Jul 21 06:19:22.017407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:22.012881 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 06:19:22.017407 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:19:22.015597 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7150569a_b586_4793_a185_e889dee4ee13.slice/crio-a7fc40e88120db4b2dd4fb29fbd9777f6d5563a72d431ede6539eb3fd2949697 WatchSource:0}: Error finding container a7fc40e88120db4b2dd4fb29fbd9777f6d5563a72d431ede6539eb3fd2949697: Status 404 returned error can't find the container with id a7fc40e88120db4b2dd4fb29fbd9777f6d5563a72d431ede6539eb3fd2949697 Jul 21 06:19:22.056349 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:22.056249 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gqqrw" event={"ID":"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae","Type":"ContainerStarted","Data":"ddbbb91c3217699e2161f883cb93c069bb41c7038c0396946d1c60b2153f5486"} Jul 21 06:19:22.062607 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:22.062496 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" event={"ID":"c3bc626c-3b7d-4689-baa0-4c22c56360d3","Type":"ContainerStarted","Data":"48a8715b6adb65b470bb6941f18163980c7f6c29cd294d45afec341f5e683920"} Jul 21 06:19:22.070026 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:22.069342 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" event={"ID":"4eb512c1-6716-43c5-a14b-c88e0a2eaf14","Type":"ContainerStarted","Data":"22077893f15a704111db738999af7dbe5826def1bcb3e05dcb7c26734ebae363"} Jul 21 06:19:22.076102 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:22.076010 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerStarted","Data":"a7fc40e88120db4b2dd4fb29fbd9777f6d5563a72d431ede6539eb3fd2949697"} Jul 21 06:19:22.103692 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:22.101847 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5v4f7" podStartSLOduration=1.409548425 podStartE2EDuration="3.101827855s" totalImagesPullingTime="1.69227943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:22.101233592 +0000 UTC m=+92.046872247" watchObservedRunningTime="2026-07-21 06:19:22.101827855 +0000 UTC m=+92.047466513" Jul 21 06:19:22.788160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:22.788092 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:22.788346 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:22.788191 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:22.793951 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:22.793924 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:23.083241 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.083185 2563 generic.go:356] "Generic (PLEG): container finished" podID="a7ea94d1-3d5b-4f1e-a587-b09b12fedeae" containerID="ddbbb91c3217699e2161f883cb93c069bb41c7038c0396946d1c60b2153f5486" exitCode=0 Jul 21 06:19:23.084382 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.084280 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gqqrw" event={"ID":"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae","Type":"ContainerDied","Data":"ddbbb91c3217699e2161f883cb93c069bb41c7038c0396946d1c60b2153f5486"} Jul 21 06:19:23.088301 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.087748 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" event={"ID":"c3bc626c-3b7d-4689-baa0-4c22c56360d3","Type":"ContainerStarted","Data":"9fea580423079203bb5a80d139d1d231b8e722973d373eac421388423f526276"} Jul 21 06:19:23.088301 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.087810 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" event={"ID":"c3bc626c-3b7d-4689-baa0-4c22c56360d3","Type":"ContainerStarted","Data":"fd466322ceecc6a0b2a23b76038bc1c3a2ba5d220fd43db643d6e54ef4c6b61a"} Jul 21 06:19:23.092914 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.092893 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:19:23.124854 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.124810 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-7f792" podStartSLOduration=2.32872507 podStartE2EDuration="4.124794984s" totalImagesPullingTime="1.796069914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:23.123194562 +0000 UTC m=+93.068833223" watchObservedRunningTime="2026-07-21 06:19:23.124794984 +0000 UTC m=+93.070433641" Jul 21 06:19:23.541954 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.541542 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-668cc967ff-j4bdq"] Jul 21 06:19:23.548579 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.547678 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.551332 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.551295 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 21 06:19:23.551938 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.551770 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-r4z9n\"" Jul 21 06:19:23.551938 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.551806 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 21 06:19:23.552958 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.552909 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 21 06:19:23.553375 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.553359 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 21 06:19:23.559246 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.554966 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3lc8ljn1lg5fs\"" Jul 21 06:19:23.562915 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.559563 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 21 06:19:23.568549 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.568430 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-668cc967ff-j4bdq"] Jul 21 06:19:23.716699 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.716632 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.716699 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.716681 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.716927 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.716733 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71c1f691-c0f4-486e-abdd-1d88f97bef96-metrics-client-ca\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.716927 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.716768 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d44x\" (UniqueName: \"kubernetes.io/projected/71c1f691-c0f4-486e-abdd-1d88f97bef96-kube-api-access-7d44x\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.716927 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.716797 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.716927 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.716821 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-tls\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.716927 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.716861 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-grpc-tls\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.717118 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.716987 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.818162 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.818130 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.818367 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.818168 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.818367 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.818303 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71c1f691-c0f4-486e-abdd-1d88f97bef96-metrics-client-ca\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.818367 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.818355 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d44x\" (UniqueName: \"kubernetes.io/projected/71c1f691-c0f4-486e-abdd-1d88f97bef96-kube-api-access-7d44x\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.818549 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.818385 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.818549 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.818413 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-tls\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.818549 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.818438 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-grpc-tls\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.818549 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.818478 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.819210 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.819117 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71c1f691-c0f4-486e-abdd-1d88f97bef96-metrics-client-ca\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.821916 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.821846 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.821916 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.821903 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.822092 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.822029 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-grpc-tls\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.822135 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.822115 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-tls\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.822891 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.822866 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.822985 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.822898 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/71c1f691-c0f4-486e-abdd-1d88f97bef96-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.826892 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.826874 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d44x\" (UniqueName: \"kubernetes.io/projected/71c1f691-c0f4-486e-abdd-1d88f97bef96-kube-api-access-7d44x\") pod \"thanos-querier-668cc967ff-j4bdq\" (UID: \"71c1f691-c0f4-486e-abdd-1d88f97bef96\") " pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.875182 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.875147 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:23.966514 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:23.966485 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-554659c859-lmjbc" Jul 21 06:19:24.093687 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:24.093652 2563 generic.go:356] "Generic (PLEG): container finished" podID="7150569a-b586-4793-a185-e889dee4ee13" containerID="2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a" exitCode=0 Jul 21 06:19:24.094134 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:24.093688 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerDied","Data":"2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a"} Jul 21 06:19:24.096672 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:24.096646 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gqqrw" event={"ID":"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae","Type":"ContainerStarted","Data":"d4b18dc3d73472622d3b48dc1f8cbf8c6e2775f96826c4442072d0d42f033c30"} Jul 21 06:19:24.096772 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:24.096698 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gqqrw" event={"ID":"a7ea94d1-3d5b-4f1e-a587-b09b12fedeae","Type":"ContainerStarted","Data":"99282f83c12638b5e7c1380d33c7ac1afb338ebc9e0ffa8234a3c8ddc3786658"} Jul 21 06:19:25.433391 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.433335 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gqqrw" podStartSLOduration=4.378281879 podStartE2EDuration="6.433317807s" totalImagesPullingTime="1.913950387s" totalInitContainerRuntime="141.085541ms" isStatefulPod=true podCreationTimestamp="2026-07-21 06:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:24.178333563 +0000 UTC m=+94.123972245" watchObservedRunningTime="2026-07-21 06:19:25.433317807 +0000 UTC m=+95.378956464" Jul 21 06:19:25.434603 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.434157 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86c64587b6-6cd47"] Jul 21 06:19:25.460410 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.460363 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c64587b6-6cd47"] Jul 21 06:19:25.460410 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.460399 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.535116 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.535074 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-trusted-ca-bundle\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.535116 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.535113 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-oauth-config\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.535425 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.535136 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b886g\" (UniqueName: \"kubernetes.io/projected/8f1664a0-1442-46a0-b74a-53eb7a636ecb-kube-api-access-b886g\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.535425 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.535280 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-service-ca\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.535425 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.535314 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-oauth-serving-cert\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.535425 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.535389 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-config\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.535425 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.535427 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-serving-cert\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.636826 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.636788 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-service-ca\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.636826 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.636832 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-oauth-serving-cert\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.637071 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.636863 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-config\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.637071 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.636895 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-serving-cert\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.637071 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.636956 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-trusted-ca-bundle\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.637071 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.636984 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-oauth-config\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.637071 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.637009 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b886g\" (UniqueName: \"kubernetes.io/projected/8f1664a0-1442-46a0-b74a-53eb7a636ecb-kube-api-access-b886g\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.637616 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.637595 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-oauth-serving-cert\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.637729 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.637707 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-config\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.637779 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.637707 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-service-ca\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.638188 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.638164 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-trusted-ca-bundle\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.639991 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.639971 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-oauth-config\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.640091 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.639973 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-serving-cert\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.645421 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.645384 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b886g\" (UniqueName: \"kubernetes.io/projected/8f1664a0-1442-46a0-b74a-53eb7a636ecb-kube-api-access-b886g\") pod \"console-86c64587b6-6cd47\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:25.772426 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:25.772348 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:26.567787 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:26.567751 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c64587b6-6cd47"] Jul 21 06:19:26.569791 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:19:26.569762 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f1664a0_1442_46a0_b74a_53eb7a636ecb.slice/crio-89168da542cdc45f9967b4c87384b527b96f623c8dc917e8b54aeb9007f996b1 WatchSource:0}: Error finding container 89168da542cdc45f9967b4c87384b527b96f623c8dc917e8b54aeb9007f996b1: Status 404 returned error can't find the container with id 89168da542cdc45f9967b4c87384b527b96f623c8dc917e8b54aeb9007f996b1 Jul 21 06:19:26.590011 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:26.589991 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-668cc967ff-j4bdq"] Jul 21 06:19:26.590732 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:19:26.590696 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71c1f691_c0f4_486e_abdd_1d88f97bef96.slice/crio-2916068983adc7258879ab1aadfca0611a6a8998cba67f46f6e73e512f64730b WatchSource:0}: Error finding container 2916068983adc7258879ab1aadfca0611a6a8998cba67f46f6e73e512f64730b: Status 404 returned error can't find the container with id 2916068983adc7258879ab1aadfca0611a6a8998cba67f46f6e73e512f64730b Jul 21 06:19:27.110026 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:27.109995 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" event={"ID":"71c1f691-c0f4-486e-abdd-1d88f97bef96","Type":"ContainerStarted","Data":"2916068983adc7258879ab1aadfca0611a6a8998cba67f46f6e73e512f64730b"} Jul 21 06:19:27.112058 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:27.112032 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zzv9d" event={"ID":"05fea74d-5017-4bf7-b70b-70919bb2c4a6","Type":"ContainerStarted","Data":"2a9ea1aaeadb1a610fcd6b9deaf3aa8b226be0bdaf5c3c53ccd3807190b60f12"} Jul 21 06:19:27.113705 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:27.113684 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c64587b6-6cd47" event={"ID":"8f1664a0-1442-46a0-b74a-53eb7a636ecb","Type":"ContainerStarted","Data":"1efde7a320fe03337e7c71d15f01798b1fd7de1976f5d7b362dddb7dd688e9d7"} Jul 21 06:19:27.113820 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:27.113728 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c64587b6-6cd47" event={"ID":"8f1664a0-1442-46a0-b74a-53eb7a636ecb","Type":"ContainerStarted","Data":"89168da542cdc45f9967b4c87384b527b96f623c8dc917e8b54aeb9007f996b1"} Jul 21 06:19:27.133247 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:27.133177 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86c64587b6-6cd47" podStartSLOduration=2.133159948 podStartE2EDuration="2.133159948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:19:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:27.131078944 +0000 UTC m=+97.076717603" watchObservedRunningTime="2026-07-21 06:19:27.133159948 +0000 UTC m=+97.078798597" Jul 21 06:19:28.125324 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:28.125282 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerStarted","Data":"03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355"} Jul 21 06:19:28.125750 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:28.125372 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerStarted","Data":"d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9"} Jul 21 06:19:28.125750 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:28.125417 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerStarted","Data":"6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b"} Jul 21 06:19:28.125750 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:28.125454 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerStarted","Data":"0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9"} Jul 21 06:19:28.125750 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:28.125491 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerStarted","Data":"3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384"} Jul 21 06:19:31.143157 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:31.143131 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zzv9d" event={"ID":"05fea74d-5017-4bf7-b70b-70919bb2c4a6","Type":"ContainerStarted","Data":"32a08ec123504836c14d22e8b2846c0504d3875d077b3e5d08c73ccefe348eef"} Jul 21 06:19:31.147072 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:31.147047 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerStarted","Data":"100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1"} Jul 21 06:19:31.149682 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:31.149656 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" event={"ID":"71c1f691-c0f4-486e-abdd-1d88f97bef96","Type":"ContainerStarted","Data":"22b42aa6e2e73e20b707f4f6a22b9adc04534698f8a0c72311792a9e445d6d93"} Jul 21 06:19:31.149759 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:31.149728 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" event={"ID":"71c1f691-c0f4-486e-abdd-1d88f97bef96","Type":"ContainerStarted","Data":"9d8ea2f56a86854867d5c4c6f7c317126b5e03b722c8ca0e93daed7e42193e2e"} Jul 21 06:19:31.149803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:31.149771 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" event={"ID":"71c1f691-c0f4-486e-abdd-1d88f97bef96","Type":"ContainerStarted","Data":"383194db52a63fd288e8f431b8f76343f63379daff75d2bcc9b51fc873e2df7c"} Jul 21 06:19:31.149862 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:31.149820 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" event={"ID":"71c1f691-c0f4-486e-abdd-1d88f97bef96","Type":"ContainerStarted","Data":"08894da0cc7c202d33f6a8d7e7080468c6c52a8364f1f771c3a1b3e0137e40a2"} Jul 21 06:19:31.177742 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:31.177704 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.323142635 podStartE2EDuration="11.177690989s" totalImagesPullingTime="7.833585154s" totalInitContainerRuntime="20.9632ms" isStatefulPod=false podCreationTimestamp="2026-07-21 06:19:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:31.176736135 +0000 UTC m=+101.122374814" watchObservedRunningTime="2026-07-21 06:19:31.177690989 +0000 UTC m=+101.123329646" Jul 21 06:19:32.157791 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:32.157750 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" event={"ID":"71c1f691-c0f4-486e-abdd-1d88f97bef96","Type":"ContainerStarted","Data":"1f30862a3974c6f309bbab97ed5cbff07c8dfddae08ceb957e67741756178217"} Jul 21 06:19:32.158190 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:32.157898 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" event={"ID":"71c1f691-c0f4-486e-abdd-1d88f97bef96","Type":"ContainerStarted","Data":"0296902f76d50c55e8f119dd218e4ae6f205ce7f9c18637ec1dcd23231e8046a"} Jul 21 06:19:32.158337 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:32.158310 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:32.182755 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:32.182717 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" podStartSLOduration=4.945048145 podStartE2EDuration="9.182702573s" totalImagesPullingTime="4.237654428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:32.179808261 +0000 UTC m=+102.125446920" watchObservedRunningTime="2026-07-21 06:19:32.182702573 +0000 UTC m=+102.128341230" Jul 21 06:19:32.958444 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:32.958414 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5sldl" Jul 21 06:19:35.171975 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:35.171946 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zzv9d" event={"ID":"05fea74d-5017-4bf7-b70b-70919bb2c4a6","Type":"ContainerStarted","Data":"3a980bd8ad2c93a82f795093a11fe6070895e148e56dede3a4ffef3113f44be3"} Jul 21 06:19:35.773107 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:35.773075 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:35.773294 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:35.773167 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:35.778921 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:35.778899 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:35.798093 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:35.798053 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zzv9d" podStartSLOduration=17.632244912 podStartE2EDuration="31.798039492s" totalImagesPullingTime="14.16579458s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 06:19:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:35.18993898 +0000 UTC m=+105.135577637" watchObservedRunningTime="2026-07-21 06:19:35.798039492 +0000 UTC m=+105.743678149" Jul 21 06:19:36.068462 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.068376 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86c64587b6-6cd47"] Jul 21 06:19:36.097390 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.097359 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5465f4d6f8-zhgjj"] Jul 21 06:19:36.118184 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.118151 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5465f4d6f8-zhgjj"] Jul 21 06:19:36.118337 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.118274 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.180429 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.180403 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:19:36.235868 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.235836 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-oauth-serving-cert\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.235868 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.235869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zll4\" (UniqueName: \"kubernetes.io/projected/4de1c2c9-85cd-442b-9dab-ebd549d484d3-kube-api-access-4zll4\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.236052 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.235896 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-serving-cert\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.236052 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.235916 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-config\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.236052 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.235968 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-oauth-config\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.236052 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.235995 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-service-ca\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.236174 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.236109 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-trusted-ca-bundle\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.337230 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.337192 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-service-ca\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.337372 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.337263 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-trusted-ca-bundle\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.337424 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.337377 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-oauth-serving-cert\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.337424 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.337409 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zll4\" (UniqueName: \"kubernetes.io/projected/4de1c2c9-85cd-442b-9dab-ebd549d484d3-kube-api-access-4zll4\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.337530 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.337465 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-serving-cert\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.337530 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.337489 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-config\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.337639 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.337549 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-oauth-config\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.338004 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.337974 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-service-ca\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.338180 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.338151 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-trusted-ca-bundle\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.338382 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.338360 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-oauth-serving-cert\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.338515 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.338489 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-config\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.340058 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.340036 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-oauth-config\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.340432 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.340413 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-serving-cert\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.345450 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.345431 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zll4\" (UniqueName: \"kubernetes.io/projected/4de1c2c9-85cd-442b-9dab-ebd549d484d3-kube-api-access-4zll4\") pod \"console-5465f4d6f8-zhgjj\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.430135 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.430093 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:36.585687 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:36.585631 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5465f4d6f8-zhgjj"] Jul 21 06:19:36.588300 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:19:36.588246 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de1c2c9_85cd_442b_9dab_ebd549d484d3.slice/crio-5178c71c8c22c1940ce294e90540d71b77aefb48d7f9ded688ae33f3abdeea75 WatchSource:0}: Error finding container 5178c71c8c22c1940ce294e90540d71b77aefb48d7f9ded688ae33f3abdeea75: Status 404 returned error can't find the container with id 5178c71c8c22c1940ce294e90540d71b77aefb48d7f9ded688ae33f3abdeea75 Jul 21 06:19:37.181952 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:37.181920 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5465f4d6f8-zhgjj" event={"ID":"4de1c2c9-85cd-442b-9dab-ebd549d484d3","Type":"ContainerStarted","Data":"5415d18c45adea897adf687f91a7a2374e5719940e812f6b258bf133a881ed80"} Jul 21 06:19:37.182547 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:37.181973 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5465f4d6f8-zhgjj" event={"ID":"4de1c2c9-85cd-442b-9dab-ebd549d484d3","Type":"ContainerStarted","Data":"5178c71c8c22c1940ce294e90540d71b77aefb48d7f9ded688ae33f3abdeea75"} Jul 21 06:19:37.201053 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:37.201007 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5465f4d6f8-zhgjj" podStartSLOduration=1.200990831 podStartE2EDuration="1.200990831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:19:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:19:37.198613364 +0000 UTC m=+107.144252023" watchObservedRunningTime="2026-07-21 06:19:37.200990831 +0000 UTC m=+107.146629489" Jul 21 06:19:38.167977 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:38.167949 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-668cc967ff-j4bdq" Jul 21 06:19:46.430481 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:46.430437 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:46.430903 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:46.430540 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:46.435327 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:46.435305 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:47.224711 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:47.224684 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:19:47.270471 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:19:47.270438 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f45bbdb99-rphzc"] Jul 21 06:20:02.202668 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:02.202603 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-86c64587b6-6cd47" podUID="8f1664a0-1442-46a0-b74a-53eb7a636ecb" containerName="console" containerID="cri-o://1efde7a320fe03337e7c71d15f01798b1fd7de1976f5d7b362dddb7dd688e9d7" gracePeriod=15 Jul 21 06:20:02.943127 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:02.943100 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:20:03.073233 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.073190 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-service-ca\") pod \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " Jul 21 06:20:03.073385 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.073286 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-oauth-serving-cert\") pod \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " Jul 21 06:20:03.073385 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.073339 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-serving-cert\") pod \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " Jul 21 06:20:03.073454 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.073387 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-trusted-ca-bundle\") pod \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " Jul 21 06:20:03.073454 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.073434 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b886g\" (UniqueName: \"kubernetes.io/projected/8f1664a0-1442-46a0-b74a-53eb7a636ecb-kube-api-access-b886g\") pod \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " Jul 21 06:20:03.073530 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.073457 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-oauth-config\") pod \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " Jul 21 06:20:03.073530 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.073488 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-config\") pod \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\" (UID: \"8f1664a0-1442-46a0-b74a-53eb7a636ecb\") " Jul 21 06:20:03.073940 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.073610 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-service-ca" (OuterVolumeSpecName: "service-ca") pod "8f1664a0-1442-46a0-b74a-53eb7a636ecb" (UID: "8f1664a0-1442-46a0-b74a-53eb7a636ecb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:20:03.073940 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.073759 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-service-ca\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:03.073940 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.073795 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8f1664a0-1442-46a0-b74a-53eb7a636ecb" (UID: "8f1664a0-1442-46a0-b74a-53eb7a636ecb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:20:03.074135 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.073947 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8f1664a0-1442-46a0-b74a-53eb7a636ecb" (UID: "8f1664a0-1442-46a0-b74a-53eb7a636ecb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:20:03.074135 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.073994 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-config" (OuterVolumeSpecName: "console-config") pod "8f1664a0-1442-46a0-b74a-53eb7a636ecb" (UID: "8f1664a0-1442-46a0-b74a-53eb7a636ecb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:20:03.075823 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.075790 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f1664a0-1442-46a0-b74a-53eb7a636ecb-kube-api-access-b886g" (OuterVolumeSpecName: "kube-api-access-b886g") pod "8f1664a0-1442-46a0-b74a-53eb7a636ecb" (UID: "8f1664a0-1442-46a0-b74a-53eb7a636ecb"). InnerVolumeSpecName "kube-api-access-b886g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:20:03.075823 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.075812 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8f1664a0-1442-46a0-b74a-53eb7a636ecb" (UID: "8f1664a0-1442-46a0-b74a-53eb7a636ecb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:20:03.075969 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.075860 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8f1664a0-1442-46a0-b74a-53eb7a636ecb" (UID: "8f1664a0-1442-46a0-b74a-53eb7a636ecb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:20:03.174765 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.174732 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-serving-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:03.174765 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.174758 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-trusted-ca-bundle\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:03.174765 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.174767 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b886g\" (UniqueName: \"kubernetes.io/projected/8f1664a0-1442-46a0-b74a-53eb7a636ecb-kube-api-access-b886g\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:03.174985 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.174777 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-oauth-config\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:03.174985 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.174787 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-console-config\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:03.174985 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.174795 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8f1664a0-1442-46a0-b74a-53eb7a636ecb-oauth-serving-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:03.285401 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.285370 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f1664a0-1442-46a0-b74a-53eb7a636ecb" containerID="1efde7a320fe03337e7c71d15f01798b1fd7de1976f5d7b362dddb7dd688e9d7" exitCode=0 Jul 21 06:20:03.285807 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.285415 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c64587b6-6cd47" Jul 21 06:20:03.285807 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.285464 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c64587b6-6cd47" event={"ID":"8f1664a0-1442-46a0-b74a-53eb7a636ecb","Type":"ContainerDied","Data":"1efde7a320fe03337e7c71d15f01798b1fd7de1976f5d7b362dddb7dd688e9d7"} Jul 21 06:20:03.285807 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.285512 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c64587b6-6cd47" event={"ID":"8f1664a0-1442-46a0-b74a-53eb7a636ecb","Type":"ContainerDied","Data":"89168da542cdc45f9967b4c87384b527b96f623c8dc917e8b54aeb9007f996b1"} Jul 21 06:20:03.285807 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.285532 2563 scope.go:117] "RemoveContainer" containerID="1efde7a320fe03337e7c71d15f01798b1fd7de1976f5d7b362dddb7dd688e9d7" Jul 21 06:20:03.295724 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.295706 2563 scope.go:117] "RemoveContainer" containerID="1efde7a320fe03337e7c71d15f01798b1fd7de1976f5d7b362dddb7dd688e9d7" Jul 21 06:20:03.296005 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:20:03.295986 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1efde7a320fe03337e7c71d15f01798b1fd7de1976f5d7b362dddb7dd688e9d7\": container with ID starting with 1efde7a320fe03337e7c71d15f01798b1fd7de1976f5d7b362dddb7dd688e9d7 not found: ID does not exist" containerID="1efde7a320fe03337e7c71d15f01798b1fd7de1976f5d7b362dddb7dd688e9d7" Jul 21 06:20:03.296058 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.296018 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1efde7a320fe03337e7c71d15f01798b1fd7de1976f5d7b362dddb7dd688e9d7"} err="failed to get container status \"1efde7a320fe03337e7c71d15f01798b1fd7de1976f5d7b362dddb7dd688e9d7\": rpc error: code = NotFound desc = could not find container \"1efde7a320fe03337e7c71d15f01798b1fd7de1976f5d7b362dddb7dd688e9d7\": container with ID starting with 1efde7a320fe03337e7c71d15f01798b1fd7de1976f5d7b362dddb7dd688e9d7 not found: ID does not exist" Jul 21 06:20:03.305924 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.305895 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86c64587b6-6cd47"] Jul 21 06:20:03.309296 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:03.309274 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86c64587b6-6cd47"] Jul 21 06:20:04.485760 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:04.485728 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f1664a0-1442-46a0-b74a-53eb7a636ecb" path="/var/lib/kubelet/pods/8f1664a0-1442-46a0-b74a-53eb7a636ecb/volumes" Jul 21 06:20:12.291142 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.291090 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f45bbdb99-rphzc" podUID="2d9b87fc-21c4-4340-b8c3-8895cb620ad1" containerName="console" containerID="cri-o://dcec55f15c09ed7e83e27e4f4b0fc5d19b9bbbbcb4b7349ccbd461594fb24b03" gracePeriod=15 Jul 21 06:20:12.542629 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.542575 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:20:12.659473 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.659442 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-config\") pod \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " Jul 21 06:20:12.659627 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.659483 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-oauth-config\") pod \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " Jul 21 06:20:12.659627 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.659510 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-service-ca\") pod \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " Jul 21 06:20:12.659700 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.659628 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8jm4\" (UniqueName: \"kubernetes.io/projected/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-kube-api-access-h8jm4\") pod \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " Jul 21 06:20:12.659700 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.659689 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-oauth-serving-cert\") pod \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " Jul 21 06:20:12.659796 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.659713 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-serving-cert\") pod \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " Jul 21 06:20:12.659796 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.659728 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-trusted-ca-bundle\") pod \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\" (UID: \"2d9b87fc-21c4-4340-b8c3-8895cb620ad1\") " Jul 21 06:20:12.659890 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.659816 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-config" (OuterVolumeSpecName: "console-config") pod "2d9b87fc-21c4-4340-b8c3-8895cb620ad1" (UID: "2d9b87fc-21c4-4340-b8c3-8895cb620ad1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:20:12.659890 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.659869 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-service-ca" (OuterVolumeSpecName: "service-ca") pod "2d9b87fc-21c4-4340-b8c3-8895cb620ad1" (UID: "2d9b87fc-21c4-4340-b8c3-8895cb620ad1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:20:12.660112 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.660085 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2d9b87fc-21c4-4340-b8c3-8895cb620ad1" (UID: "2d9b87fc-21c4-4340-b8c3-8895cb620ad1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:20:12.660316 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.660120 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-config\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:12.660316 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.660137 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-service-ca\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:12.660316 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.660200 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2d9b87fc-21c4-4340-b8c3-8895cb620ad1" (UID: "2d9b87fc-21c4-4340-b8c3-8895cb620ad1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:20:12.661926 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.661903 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2d9b87fc-21c4-4340-b8c3-8895cb620ad1" (UID: "2d9b87fc-21c4-4340-b8c3-8895cb620ad1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:20:12.662384 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.662364 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2d9b87fc-21c4-4340-b8c3-8895cb620ad1" (UID: "2d9b87fc-21c4-4340-b8c3-8895cb620ad1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:20:12.662448 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.662426 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-kube-api-access-h8jm4" (OuterVolumeSpecName: "kube-api-access-h8jm4") pod "2d9b87fc-21c4-4340-b8c3-8895cb620ad1" (UID: "2d9b87fc-21c4-4340-b8c3-8895cb620ad1"). InnerVolumeSpecName "kube-api-access-h8jm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:20:12.761446 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.761406 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-oauth-serving-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:12.761446 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.761443 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-serving-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:12.761446 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.761453 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-trusted-ca-bundle\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:12.761446 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.761461 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-console-oauth-config\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:12.761668 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:12.761469 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8jm4\" (UniqueName: \"kubernetes.io/projected/2d9b87fc-21c4-4340-b8c3-8895cb620ad1-kube-api-access-h8jm4\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:13.325106 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:13.325069 2563 generic.go:356] "Generic (PLEG): container finished" podID="2d9b87fc-21c4-4340-b8c3-8895cb620ad1" containerID="dcec55f15c09ed7e83e27e4f4b0fc5d19b9bbbbcb4b7349ccbd461594fb24b03" exitCode=0 Jul 21 06:20:13.325570 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:13.325120 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f45bbdb99-rphzc" Jul 21 06:20:13.325570 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:13.325125 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f45bbdb99-rphzc" event={"ID":"2d9b87fc-21c4-4340-b8c3-8895cb620ad1","Type":"ContainerDied","Data":"dcec55f15c09ed7e83e27e4f4b0fc5d19b9bbbbcb4b7349ccbd461594fb24b03"} Jul 21 06:20:13.325570 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:13.325175 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f45bbdb99-rphzc" event={"ID":"2d9b87fc-21c4-4340-b8c3-8895cb620ad1","Type":"ContainerDied","Data":"16c3c0fe8f8c37d697b872d2acc7c38953ba1d16345b4dc987113de670e4c5ec"} Jul 21 06:20:13.325570 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:13.325202 2563 scope.go:117] "RemoveContainer" containerID="dcec55f15c09ed7e83e27e4f4b0fc5d19b9bbbbcb4b7349ccbd461594fb24b03" Jul 21 06:20:13.333709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:13.333691 2563 scope.go:117] "RemoveContainer" containerID="dcec55f15c09ed7e83e27e4f4b0fc5d19b9bbbbcb4b7349ccbd461594fb24b03" Jul 21 06:20:13.333953 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:20:13.333934 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcec55f15c09ed7e83e27e4f4b0fc5d19b9bbbbcb4b7349ccbd461594fb24b03\": container with ID starting with dcec55f15c09ed7e83e27e4f4b0fc5d19b9bbbbcb4b7349ccbd461594fb24b03 not found: ID does not exist" containerID="dcec55f15c09ed7e83e27e4f4b0fc5d19b9bbbbcb4b7349ccbd461594fb24b03" Jul 21 06:20:13.333997 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:13.333961 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcec55f15c09ed7e83e27e4f4b0fc5d19b9bbbbcb4b7349ccbd461594fb24b03"} err="failed to get container status \"dcec55f15c09ed7e83e27e4f4b0fc5d19b9bbbbcb4b7349ccbd461594fb24b03\": rpc error: code = NotFound desc = could not find container \"dcec55f15c09ed7e83e27e4f4b0fc5d19b9bbbbcb4b7349ccbd461594fb24b03\": container with ID starting with dcec55f15c09ed7e83e27e4f4b0fc5d19b9bbbbcb4b7349ccbd461594fb24b03 not found: ID does not exist" Jul 21 06:20:13.349939 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:13.347493 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f45bbdb99-rphzc"] Jul 21 06:20:13.350270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:13.350250 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f45bbdb99-rphzc"] Jul 21 06:20:14.485396 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:14.485364 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d9b87fc-21c4-4340-b8c3-8895cb620ad1" path="/var/lib/kubelet/pods/2d9b87fc-21c4-4340-b8c3-8895cb620ad1/volumes" Jul 21 06:20:39.848051 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:39.848015 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 06:20:39.848564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:39.848328 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="alertmanager" containerID="cri-o://3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384" gracePeriod=120 Jul 21 06:20:39.848564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:39.848405 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="kube-rbac-proxy-metric" containerID="cri-o://03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355" gracePeriod=120 Jul 21 06:20:39.848564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:39.848433 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="prom-label-proxy" containerID="cri-o://100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1" gracePeriod=120 Jul 21 06:20:39.848564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:39.848457 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="kube-rbac-proxy-web" containerID="cri-o://6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b" gracePeriod=120 Jul 21 06:20:39.848564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:39.848466 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="config-reloader" containerID="cri-o://0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9" gracePeriod=120 Jul 21 06:20:39.848564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:39.848553 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="kube-rbac-proxy" containerID="cri-o://d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9" gracePeriod=120 Jul 21 06:20:40.432179 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:40.432149 2563 generic.go:356] "Generic (PLEG): container finished" podID="7150569a-b586-4793-a185-e889dee4ee13" containerID="100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1" exitCode=0 Jul 21 06:20:40.432179 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:40.432175 2563 generic.go:356] "Generic (PLEG): container finished" podID="7150569a-b586-4793-a185-e889dee4ee13" containerID="d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9" exitCode=0 Jul 21 06:20:40.432179 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:40.432185 2563 generic.go:356] "Generic (PLEG): container finished" podID="7150569a-b586-4793-a185-e889dee4ee13" containerID="0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9" exitCode=0 Jul 21 06:20:40.432431 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:40.432192 2563 generic.go:356] "Generic (PLEG): container finished" podID="7150569a-b586-4793-a185-e889dee4ee13" containerID="3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384" exitCode=0 Jul 21 06:20:40.432431 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:40.432237 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerDied","Data":"100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1"} Jul 21 06:20:40.432431 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:40.432299 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerDied","Data":"d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9"} Jul 21 06:20:40.432431 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:40.432325 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerDied","Data":"0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9"} Jul 21 06:20:40.432431 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:40.432349 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerDied","Data":"3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384"} Jul 21 06:20:41.085083 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.085061 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.185966 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.185892 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7150569a-b586-4793-a185-e889dee4ee13-alertmanager-trusted-ca-bundle\") pod \"7150569a-b586-4793-a185-e889dee4ee13\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " Jul 21 06:20:41.185966 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.185933 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-main-tls\") pod \"7150569a-b586-4793-a185-e889dee4ee13\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " Jul 21 06:20:41.186152 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.185968 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7150569a-b586-4793-a185-e889dee4ee13-config-out\") pod \"7150569a-b586-4793-a185-e889dee4ee13\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " Jul 21 06:20:41.186152 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.186006 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7150569a-b586-4793-a185-e889dee4ee13-alertmanager-main-db\") pod \"7150569a-b586-4793-a185-e889dee4ee13\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " Jul 21 06:20:41.186152 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.186030 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-config-volume\") pod \"7150569a-b586-4793-a185-e889dee4ee13\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " Jul 21 06:20:41.186152 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.186045 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-cluster-tls-config\") pod \"7150569a-b586-4793-a185-e889dee4ee13\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " Jul 21 06:20:41.186152 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.186073 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-web-config\") pod \"7150569a-b586-4793-a185-e889dee4ee13\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " Jul 21 06:20:41.186152 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.186097 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy-web\") pod \"7150569a-b586-4793-a185-e889dee4ee13\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " Jul 21 06:20:41.186152 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.186143 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy\") pod \"7150569a-b586-4793-a185-e889dee4ee13\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " Jul 21 06:20:41.186536 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.186176 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gg9wz\" (UniqueName: \"kubernetes.io/projected/7150569a-b586-4793-a185-e889dee4ee13-kube-api-access-gg9wz\") pod \"7150569a-b586-4793-a185-e889dee4ee13\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " Jul 21 06:20:41.186536 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.186204 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy-metric\") pod \"7150569a-b586-4793-a185-e889dee4ee13\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " Jul 21 06:20:41.186536 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.186264 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7150569a-b586-4793-a185-e889dee4ee13-metrics-client-ca\") pod \"7150569a-b586-4793-a185-e889dee4ee13\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " Jul 21 06:20:41.186536 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.186338 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7150569a-b586-4793-a185-e889dee4ee13-tls-assets\") pod \"7150569a-b586-4793-a185-e889dee4ee13\" (UID: \"7150569a-b586-4793-a185-e889dee4ee13\") " Jul 21 06:20:41.186536 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.186345 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7150569a-b586-4793-a185-e889dee4ee13-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "7150569a-b586-4793-a185-e889dee4ee13" (UID: "7150569a-b586-4793-a185-e889dee4ee13"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:20:41.186817 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.186592 2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7150569a-b586-4793-a185-e889dee4ee13-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:41.187234 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.187181 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7150569a-b586-4793-a185-e889dee4ee13-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "7150569a-b586-4793-a185-e889dee4ee13" (UID: "7150569a-b586-4793-a185-e889dee4ee13"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:20:41.187788 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.187758 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7150569a-b586-4793-a185-e889dee4ee13-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "7150569a-b586-4793-a185-e889dee4ee13" (UID: "7150569a-b586-4793-a185-e889dee4ee13"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:20:41.189014 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.188987 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7150569a-b586-4793-a185-e889dee4ee13-config-out" (OuterVolumeSpecName: "config-out") pod "7150569a-b586-4793-a185-e889dee4ee13" (UID: "7150569a-b586-4793-a185-e889dee4ee13"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:20:41.189496 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.189455 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7150569a-b586-4793-a185-e889dee4ee13-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7150569a-b586-4793-a185-e889dee4ee13" (UID: "7150569a-b586-4793-a185-e889dee4ee13"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:20:41.189589 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.189556 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "7150569a-b586-4793-a185-e889dee4ee13" (UID: "7150569a-b586-4793-a185-e889dee4ee13"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:20:41.189647 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.189603 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-config-volume" (OuterVolumeSpecName: "config-volume") pod "7150569a-b586-4793-a185-e889dee4ee13" (UID: "7150569a-b586-4793-a185-e889dee4ee13"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:20:41.189751 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.189724 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "7150569a-b586-4793-a185-e889dee4ee13" (UID: "7150569a-b586-4793-a185-e889dee4ee13"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:20:41.190084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.190058 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7150569a-b586-4793-a185-e889dee4ee13-kube-api-access-gg9wz" (OuterVolumeSpecName: "kube-api-access-gg9wz") pod "7150569a-b586-4793-a185-e889dee4ee13" (UID: "7150569a-b586-4793-a185-e889dee4ee13"). InnerVolumeSpecName "kube-api-access-gg9wz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:20:41.190155 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.190094 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "7150569a-b586-4793-a185-e889dee4ee13" (UID: "7150569a-b586-4793-a185-e889dee4ee13"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:20:41.191729 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.191707 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "7150569a-b586-4793-a185-e889dee4ee13" (UID: "7150569a-b586-4793-a185-e889dee4ee13"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:20:41.194613 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.194587 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "7150569a-b586-4793-a185-e889dee4ee13" (UID: "7150569a-b586-4793-a185-e889dee4ee13"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:20:41.200857 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.200838 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-web-config" (OuterVolumeSpecName: "web-config") pod "7150569a-b586-4793-a185-e889dee4ee13" (UID: "7150569a-b586-4793-a185-e889dee4ee13"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:20:41.287341 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.287293 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-main-tls\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:41.287341 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.287314 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7150569a-b586-4793-a185-e889dee4ee13-config-out\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:41.287341 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.287323 2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7150569a-b586-4793-a185-e889dee4ee13-alertmanager-main-db\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:41.287341 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.287331 2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-config-volume\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:41.287341 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.287340 2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-cluster-tls-config\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:41.287539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.287348 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-web-config\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:41.287539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.287358 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:41.287539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.287367 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:41.287539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.287375 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg9wz\" (UniqueName: \"kubernetes.io/projected/7150569a-b586-4793-a185-e889dee4ee13-kube-api-access-gg9wz\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:41.287539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.287383 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7150569a-b586-4793-a185-e889dee4ee13-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:41.287539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.287392 2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7150569a-b586-4793-a185-e889dee4ee13-metrics-client-ca\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:41.287539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.287400 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7150569a-b586-4793-a185-e889dee4ee13-tls-assets\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:20:41.443733 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.443648 2563 generic.go:356] "Generic (PLEG): container finished" podID="7150569a-b586-4793-a185-e889dee4ee13" containerID="03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355" exitCode=0 Jul 21 06:20:41.443733 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.443673 2563 generic.go:356] "Generic (PLEG): container finished" podID="7150569a-b586-4793-a185-e889dee4ee13" containerID="6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b" exitCode=0 Jul 21 06:20:41.443733 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.443728 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerDied","Data":"03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355"} Jul 21 06:20:41.443986 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.443759 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.443986 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.443811 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerDied","Data":"6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b"} Jul 21 06:20:41.443986 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.443837 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7150569a-b586-4793-a185-e889dee4ee13","Type":"ContainerDied","Data":"a7fc40e88120db4b2dd4fb29fbd9777f6d5563a72d431ede6539eb3fd2949697"} Jul 21 06:20:41.443986 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.443866 2563 scope.go:117] "RemoveContainer" containerID="100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1" Jul 21 06:20:41.451923 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.451904 2563 scope.go:117] "RemoveContainer" containerID="03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355" Jul 21 06:20:41.459720 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.459704 2563 scope.go:117] "RemoveContainer" containerID="d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9" Jul 21 06:20:41.466271 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.466247 2563 scope.go:117] "RemoveContainer" containerID="6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b" Jul 21 06:20:41.469386 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.469362 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 06:20:41.473042 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.473023 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 06:20:41.473933 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.473916 2563 scope.go:117] "RemoveContainer" containerID="0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9" Jul 21 06:20:41.480674 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.480661 2563 scope.go:117] "RemoveContainer" containerID="3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384" Jul 21 06:20:41.487444 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.487405 2563 scope.go:117] "RemoveContainer" containerID="2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a" Jul 21 06:20:41.494162 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.494147 2563 scope.go:117] "RemoveContainer" containerID="100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1" Jul 21 06:20:41.494426 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:20:41.494407 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1\": container with ID starting with 100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1 not found: ID does not exist" containerID="100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1" Jul 21 06:20:41.494495 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.494434 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1"} err="failed to get container status \"100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1\": rpc error: code = NotFound desc = could not find container \"100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1\": container with ID starting with 100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1 not found: ID does not exist" Jul 21 06:20:41.494495 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.494450 2563 scope.go:117] "RemoveContainer" containerID="03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355" Jul 21 06:20:41.494735 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:20:41.494712 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355\": container with ID starting with 03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355 not found: ID does not exist" containerID="03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355" Jul 21 06:20:41.494910 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.494741 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355"} err="failed to get container status \"03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355\": rpc error: code = NotFound desc = could not find container \"03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355\": container with ID starting with 03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355 not found: ID does not exist" Jul 21 06:20:41.494910 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.494769 2563 scope.go:117] "RemoveContainer" containerID="d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9" Jul 21 06:20:41.495235 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:20:41.495196 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9\": container with ID starting with d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9 not found: ID does not exist" containerID="d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9" Jul 21 06:20:41.495324 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.495257 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9"} err="failed to get container status \"d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9\": rpc error: code = NotFound desc = could not find container \"d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9\": container with ID starting with d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9 not found: ID does not exist" Jul 21 06:20:41.495324 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.495279 2563 scope.go:117] "RemoveContainer" containerID="6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b" Jul 21 06:20:41.495581 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:20:41.495558 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b\": container with ID starting with 6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b not found: ID does not exist" containerID="6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b" Jul 21 06:20:41.495705 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.495587 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b"} err="failed to get container status \"6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b\": rpc error: code = NotFound desc = could not find container \"6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b\": container with ID starting with 6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b not found: ID does not exist" Jul 21 06:20:41.495705 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.495608 2563 scope.go:117] "RemoveContainer" containerID="0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9" Jul 21 06:20:41.495984 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:20:41.495963 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9\": container with ID starting with 0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9 not found: ID does not exist" containerID="0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9" Jul 21 06:20:41.496065 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.495990 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9"} err="failed to get container status \"0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9\": rpc error: code = NotFound desc = could not find container \"0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9\": container with ID starting with 0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9 not found: ID does not exist" Jul 21 06:20:41.496065 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.496011 2563 scope.go:117] "RemoveContainer" containerID="3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384" Jul 21 06:20:41.496290 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:20:41.496270 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384\": container with ID starting with 3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384 not found: ID does not exist" containerID="3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384" Jul 21 06:20:41.496350 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.496295 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384"} err="failed to get container status \"3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384\": rpc error: code = NotFound desc = could not find container \"3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384\": container with ID starting with 3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384 not found: ID does not exist" Jul 21 06:20:41.496350 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.496309 2563 scope.go:117] "RemoveContainer" containerID="2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a" Jul 21 06:20:41.496564 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:20:41.496549 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a\": container with ID starting with 2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a not found: ID does not exist" containerID="2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a" Jul 21 06:20:41.496601 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.496567 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a"} err="failed to get container status \"2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a\": rpc error: code = NotFound desc = could not find container \"2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a\": container with ID starting with 2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a not found: ID does not exist" Jul 21 06:20:41.496601 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.496583 2563 scope.go:117] "RemoveContainer" containerID="100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1" Jul 21 06:20:41.496830 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.496813 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1"} err="failed to get container status \"100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1\": rpc error: code = NotFound desc = could not find container \"100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1\": container with ID starting with 100c4cd2c8588946b486a5cddd3b5911be4d059b458ff513ebeb57d34d7959c1 not found: ID does not exist" Jul 21 06:20:41.496893 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.496831 2563 scope.go:117] "RemoveContainer" containerID="03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355" Jul 21 06:20:41.497022 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497007 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 06:20:41.497069 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497049 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355"} err="failed to get container status \"03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355\": rpc error: code = NotFound desc = could not find container \"03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355\": container with ID starting with 03a52ceedfc70de29d40b595384c8da417643c625585175e4cfc8a59688d1355 not found: ID does not exist" Jul 21 06:20:41.497102 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497068 2563 scope.go:117] "RemoveContainer" containerID="d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9" Jul 21 06:20:41.497351 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497336 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="kube-rbac-proxy" Jul 21 06:20:41.497400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497355 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="kube-rbac-proxy" Jul 21 06:20:41.497400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497365 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="kube-rbac-proxy-metric" Jul 21 06:20:41.497400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497370 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="kube-rbac-proxy-metric" Jul 21 06:20:41.497400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497376 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9"} err="failed to get container status \"d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9\": rpc error: code = NotFound desc = could not find container \"d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9\": container with ID starting with d38f489891eac3cba4b51145a8b89b06fd254ba328e0c73a285c2c4b219f6da9 not found: ID does not exist" Jul 21 06:20:41.497400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497398 2563 scope.go:117] "RemoveContainer" containerID="6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b" Jul 21 06:20:41.497553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497381 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="kube-rbac-proxy-web" Jul 21 06:20:41.497553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497448 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="kube-rbac-proxy-web" Jul 21 06:20:41.497553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497461 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f1664a0-1442-46a0-b74a-53eb7a636ecb" containerName="console" Jul 21 06:20:41.497553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497467 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f1664a0-1442-46a0-b74a-53eb7a636ecb" containerName="console" Jul 21 06:20:41.497553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497476 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="alertmanager" Jul 21 06:20:41.497553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497483 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="alertmanager" Jul 21 06:20:41.497553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497492 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="config-reloader" Jul 21 06:20:41.497553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497500 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="config-reloader" Jul 21 06:20:41.497553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497508 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="prom-label-proxy" Jul 21 06:20:41.497553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497516 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="prom-label-proxy" Jul 21 06:20:41.497553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497534 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="init-config-reloader" Jul 21 06:20:41.497553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497542 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="init-config-reloader" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497561 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d9b87fc-21c4-4340-b8c3-8895cb620ad1" containerName="console" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497569 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d9b87fc-21c4-4340-b8c3-8895cb620ad1" containerName="console" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497642 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="alertmanager" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497652 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="kube-rbac-proxy-metric" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497660 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f1664a0-1442-46a0-b74a-53eb7a636ecb" containerName="console" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497668 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="config-reloader" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497667 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b"} err="failed to get container status \"6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b\": rpc error: code = NotFound desc = could not find container \"6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b\": container with ID starting with 6d659ec927df9b986355282d824d44a6b2c5e5d9dbddaad733a685c40836d52b not found: ID does not exist" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497682 2563 scope.go:117] "RemoveContainer" containerID="0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497676 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d9b87fc-21c4-4340-b8c3-8895cb620ad1" containerName="console" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497739 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="prom-label-proxy" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497747 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="kube-rbac-proxy" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497757 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7150569a-b586-4793-a185-e889dee4ee13" containerName="kube-rbac-proxy-web" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497929 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9"} err="failed to get container status \"0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9\": rpc error: code = NotFound desc = could not find container \"0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9\": container with ID starting with 0cd6c6b3051101baba81cb17fb16b40b063cf93ca4979ae0d74f6bb29f2d51d9 not found: ID does not exist" Jul 21 06:20:41.497948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.497943 2563 scope.go:117] "RemoveContainer" containerID="3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384" Jul 21 06:20:41.498627 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.498137 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384"} err="failed to get container status \"3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384\": rpc error: code = NotFound desc = could not find container \"3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384\": container with ID starting with 3f629ffa611ce7221e0a4a4ed2d7b9d11c23cfa5a220e646fcd15a0fbc311384 not found: ID does not exist" Jul 21 06:20:41.498627 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.498151 2563 scope.go:117] "RemoveContainer" containerID="2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a" Jul 21 06:20:41.498627 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.498390 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a"} err="failed to get container status \"2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a\": rpc error: code = NotFound desc = could not find container \"2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a\": container with ID starting with 2ce35570e6b80b628d493ed3746ac95b1a09749284aa5d47fbcb13793cc21c6a not found: ID does not exist" Jul 21 06:20:41.503102 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.503087 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.506110 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.506074 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 21 06:20:41.506262 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.506152 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 21 06:20:41.506262 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.506210 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 21 06:20:41.506389 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.506373 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 21 06:20:41.506470 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.506395 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 21 06:20:41.506538 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.506469 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 21 06:20:41.506538 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.506525 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kb6mw\"" Jul 21 06:20:41.506664 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.506574 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 21 06:20:41.506664 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.506596 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 21 06:20:41.511669 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.511644 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 21 06:20:41.514542 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.514523 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 06:20:41.590820 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.590793 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-web-config\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.590945 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.590827 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.590945 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.590847 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.590945 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.590894 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.591058 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.590960 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.591058 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.590986 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.591058 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.591015 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-config-volume\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.591058 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.591041 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-config-out\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.591058 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.591055 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6bfz\" (UniqueName: \"kubernetes.io/projected/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-kube-api-access-r6bfz\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.591261 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.591085 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.591261 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.591109 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.591327 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.591276 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.591327 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.591313 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692038 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692010 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692038 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692039 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692165 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692063 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692165 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692093 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692165 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692125 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692165 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692156 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-config-volume\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692381 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692180 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-config-out\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692381 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692203 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6bfz\" (UniqueName: \"kubernetes.io/projected/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-kube-api-access-r6bfz\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692381 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692245 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692381 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692266 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692381 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692298 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692381 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692328 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692381 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692376 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-web-config\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.692711 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692513 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.693366 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.692905 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.694532 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.694147 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.695240 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.695194 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.695240 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.695207 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-web-config\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.695390 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.695377 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.695450 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.695392 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.695509 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.695458 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.695639 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.695608 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.695814 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.695797 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-config-volume\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.696823 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.696807 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.697111 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.697092 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-config-out\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.703152 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.703136 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6bfz\" (UniqueName: \"kubernetes.io/projected/17c2cf80-79a4-4fc4-8a15-0cc6354d405a-kube-api-access-r6bfz\") pod \"alertmanager-main-0\" (UID: \"17c2cf80-79a4-4fc4-8a15-0cc6354d405a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.812838 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.812806 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 06:20:41.949029 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:41.949005 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 06:20:41.950852 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:20:41.950820 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17c2cf80_79a4_4fc4_8a15_0cc6354d405a.slice/crio-a94e262eb62639ca951b46b47a73d80a6da28e77c7fe2db9b0a81627cf362ffe WatchSource:0}: Error finding container a94e262eb62639ca951b46b47a73d80a6da28e77c7fe2db9b0a81627cf362ffe: Status 404 returned error can't find the container with id a94e262eb62639ca951b46b47a73d80a6da28e77c7fe2db9b0a81627cf362ffe Jul 21 06:20:42.449523 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:42.449491 2563 generic.go:356] "Generic (PLEG): container finished" podID="17c2cf80-79a4-4fc4-8a15-0cc6354d405a" containerID="73af4e6a5e5b6ad5638655193db4eda421c6b1f762899e6f739f9dea8798260d" exitCode=0 Jul 21 06:20:42.449911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:42.449527 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"17c2cf80-79a4-4fc4-8a15-0cc6354d405a","Type":"ContainerDied","Data":"73af4e6a5e5b6ad5638655193db4eda421c6b1f762899e6f739f9dea8798260d"} Jul 21 06:20:42.449911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:42.449575 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"17c2cf80-79a4-4fc4-8a15-0cc6354d405a","Type":"ContainerStarted","Data":"a94e262eb62639ca951b46b47a73d80a6da28e77c7fe2db9b0a81627cf362ffe"} Jul 21 06:20:42.485530 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:42.485504 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7150569a-b586-4793-a185-e889dee4ee13" path="/var/lib/kubelet/pods/7150569a-b586-4793-a185-e889dee4ee13/volumes" Jul 21 06:20:43.456949 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:43.456914 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"17c2cf80-79a4-4fc4-8a15-0cc6354d405a","Type":"ContainerStarted","Data":"e78ea236a8715290208eaa1cccdc3abb4c305b2157f66bfc3323ae6dc2cbd40a"} Jul 21 06:20:43.457407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:43.457006 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"17c2cf80-79a4-4fc4-8a15-0cc6354d405a","Type":"ContainerStarted","Data":"f562aa2a30e7bdd2859e99f3d497df4d15d9f78dbc8071ab341c379276603e40"} Jul 21 06:20:43.457407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:43.457032 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"17c2cf80-79a4-4fc4-8a15-0cc6354d405a","Type":"ContainerStarted","Data":"76e591430dcf5153921f230dda290fbbc3b5b62f5c4f17219ad750576bad1341"} Jul 21 06:20:43.457407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:43.457059 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"17c2cf80-79a4-4fc4-8a15-0cc6354d405a","Type":"ContainerStarted","Data":"6d1a0100a3f007fbee4b60562dfe1d83a457b7ce7b51400660a1df011e45c2c6"} Jul 21 06:20:43.457407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:43.457084 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"17c2cf80-79a4-4fc4-8a15-0cc6354d405a","Type":"ContainerStarted","Data":"c70d3595d1a81ef47ce3818a56bd51089b30e30d54e2b5eedd1171640c9c2722"} Jul 21 06:20:43.457407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:43.457108 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"17c2cf80-79a4-4fc4-8a15-0cc6354d405a","Type":"ContainerStarted","Data":"ee36f8ecb8e87a40cef204ab254310b60f8ffaeb8d54d59459b8df54f8e94771"} Jul 21 06:20:43.483740 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:43.483697 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.4699446849999998 podStartE2EDuration="2.483683809s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.739124ms" isStatefulPod=false podCreationTimestamp="2026-07-21 06:20:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:20:43.483327133 +0000 UTC m=+173.428965790" watchObservedRunningTime="2026-07-21 06:20:43.483683809 +0000 UTC m=+173.429322466" Jul 21 06:20:45.923817 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:45.923780 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-855fc695dd-frncg"] Jul 21 06:20:45.927171 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:45.927151 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:45.939589 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:45.939565 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-855fc695dd-frncg"] Jul 21 06:20:46.030712 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.030682 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-oauth-serving-cert\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.030712 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.030718 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-trusted-ca-bundle\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.030935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.030735 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgp7m\" (UniqueName: \"kubernetes.io/projected/9e657d37-915f-49b5-bd92-13c59b0a54b2-kube-api-access-tgp7m\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.030935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.030754 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-serving-cert\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.030935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.030836 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-service-ca\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.030935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.030873 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-config\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.030935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.030917 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-oauth-config\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.131985 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.131956 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-config\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.132138 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.131999 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-oauth-config\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.132138 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.132028 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-oauth-serving-cert\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.132264 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.132133 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-trusted-ca-bundle\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.132264 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.132160 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgp7m\" (UniqueName: \"kubernetes.io/projected/9e657d37-915f-49b5-bd92-13c59b0a54b2-kube-api-access-tgp7m\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.132264 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.132179 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-serving-cert\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.132264 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.132212 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-service-ca\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.132783 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.132753 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-oauth-serving-cert\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.132888 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.132872 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-service-ca\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.133015 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.132996 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-config\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.133162 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.133141 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-trusted-ca-bundle\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.134595 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.134572 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-oauth-config\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.134729 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.134708 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-serving-cert\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.139837 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.139816 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgp7m\" (UniqueName: \"kubernetes.io/projected/9e657d37-915f-49b5-bd92-13c59b0a54b2-kube-api-access-tgp7m\") pod \"console-855fc695dd-frncg\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.236608 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.236550 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:46.366725 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.366701 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-855fc695dd-frncg"] Jul 21 06:20:46.368633 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:20:46.368602 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e657d37_915f_49b5_bd92_13c59b0a54b2.slice/crio-cecf8aa0d05c722a83e879a6bc1e0e5bc01c7005f37940c30afd1739a2ffc669 WatchSource:0}: Error finding container cecf8aa0d05c722a83e879a6bc1e0e5bc01c7005f37940c30afd1739a2ffc669: Status 404 returned error can't find the container with id cecf8aa0d05c722a83e879a6bc1e0e5bc01c7005f37940c30afd1739a2ffc669 Jul 21 06:20:46.471175 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.471135 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855fc695dd-frncg" event={"ID":"9e657d37-915f-49b5-bd92-13c59b0a54b2","Type":"ContainerStarted","Data":"ddb412e26a22cc6a4a38c6516d87296703a2ca3dee1049660249f71f29a5f46d"} Jul 21 06:20:46.471335 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.471199 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855fc695dd-frncg" event={"ID":"9e657d37-915f-49b5-bd92-13c59b0a54b2","Type":"ContainerStarted","Data":"cecf8aa0d05c722a83e879a6bc1e0e5bc01c7005f37940c30afd1739a2ffc669"} Jul 21 06:20:46.490862 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:46.490764 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-855fc695dd-frncg" podStartSLOduration=1.490744001 podStartE2EDuration="1.490744001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:20:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:20:46.487882838 +0000 UTC m=+176.433521487" watchObservedRunningTime="2026-07-21 06:20:46.490744001 +0000 UTC m=+176.436382657" Jul 21 06:20:56.236939 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:56.236900 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:56.237349 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:56.236973 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:56.241488 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:56.241465 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:56.514519 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:56.514447 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:20:56.556821 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:20:56.556792 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5465f4d6f8-zhgjj"] Jul 21 06:21:21.579047 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.579005 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5465f4d6f8-zhgjj" podUID="4de1c2c9-85cd-442b-9dab-ebd549d484d3" containerName="console" containerID="cri-o://5415d18c45adea897adf687f91a7a2374e5719940e812f6b258bf133a881ed80" gracePeriod=15 Jul 21 06:21:21.825547 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.825524 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:21:21.934307 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.934207 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-service-ca\") pod \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " Jul 21 06:21:21.934307 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.934272 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zll4\" (UniqueName: \"kubernetes.io/projected/4de1c2c9-85cd-442b-9dab-ebd549d484d3-kube-api-access-4zll4\") pod \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " Jul 21 06:21:21.934531 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.934393 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-trusted-ca-bundle\") pod \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " Jul 21 06:21:21.934531 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.934446 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-oauth-serving-cert\") pod \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " Jul 21 06:21:21.934531 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.934485 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-config\") pod \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " Jul 21 06:21:21.934531 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.934521 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-serving-cert\") pod \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " Jul 21 06:21:21.934719 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.934556 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-oauth-config\") pod \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\" (UID: \"4de1c2c9-85cd-442b-9dab-ebd549d484d3\") " Jul 21 06:21:21.934719 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.934683 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-service-ca" (OuterVolumeSpecName: "service-ca") pod "4de1c2c9-85cd-442b-9dab-ebd549d484d3" (UID: "4de1c2c9-85cd-442b-9dab-ebd549d484d3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:21:21.934819 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.934795 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4de1c2c9-85cd-442b-9dab-ebd549d484d3" (UID: "4de1c2c9-85cd-442b-9dab-ebd549d484d3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:21:21.934874 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.934841 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4de1c2c9-85cd-442b-9dab-ebd549d484d3" (UID: "4de1c2c9-85cd-442b-9dab-ebd549d484d3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:21:21.934929 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.934880 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-service-ca\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:21:21.934929 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.934873 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-config" (OuterVolumeSpecName: "console-config") pod "4de1c2c9-85cd-442b-9dab-ebd549d484d3" (UID: "4de1c2c9-85cd-442b-9dab-ebd549d484d3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:21:21.936637 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.936606 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4de1c2c9-85cd-442b-9dab-ebd549d484d3-kube-api-access-4zll4" (OuterVolumeSpecName: "kube-api-access-4zll4") pod "4de1c2c9-85cd-442b-9dab-ebd549d484d3" (UID: "4de1c2c9-85cd-442b-9dab-ebd549d484d3"). InnerVolumeSpecName "kube-api-access-4zll4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:21:21.936754 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.936636 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4de1c2c9-85cd-442b-9dab-ebd549d484d3" (UID: "4de1c2c9-85cd-442b-9dab-ebd549d484d3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:21:21.936754 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:21.936710 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4de1c2c9-85cd-442b-9dab-ebd549d484d3" (UID: "4de1c2c9-85cd-442b-9dab-ebd549d484d3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:21:22.036288 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.036253 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zll4\" (UniqueName: \"kubernetes.io/projected/4de1c2c9-85cd-442b-9dab-ebd549d484d3-kube-api-access-4zll4\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:21:22.036288 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.036278 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-trusted-ca-bundle\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:21:22.036288 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.036287 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-oauth-serving-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:21:22.036288 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.036297 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-config\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:21:22.036548 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.036306 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-serving-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:21:22.036548 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.036316 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4de1c2c9-85cd-442b-9dab-ebd549d484d3-console-oauth-config\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:21:22.612812 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.612777 2563 generic.go:356] "Generic (PLEG): container finished" podID="4de1c2c9-85cd-442b-9dab-ebd549d484d3" containerID="5415d18c45adea897adf687f91a7a2374e5719940e812f6b258bf133a881ed80" exitCode=0 Jul 21 06:21:22.613245 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.612819 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5465f4d6f8-zhgjj" event={"ID":"4de1c2c9-85cd-442b-9dab-ebd549d484d3","Type":"ContainerDied","Data":"5415d18c45adea897adf687f91a7a2374e5719940e812f6b258bf133a881ed80"} Jul 21 06:21:22.613245 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.612861 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5465f4d6f8-zhgjj" event={"ID":"4de1c2c9-85cd-442b-9dab-ebd549d484d3","Type":"ContainerDied","Data":"5178c71c8c22c1940ce294e90540d71b77aefb48d7f9ded688ae33f3abdeea75"} Jul 21 06:21:22.613245 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.612880 2563 scope.go:117] "RemoveContainer" containerID="5415d18c45adea897adf687f91a7a2374e5719940e812f6b258bf133a881ed80" Jul 21 06:21:22.613245 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.612820 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5465f4d6f8-zhgjj" Jul 21 06:21:22.620965 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.620946 2563 scope.go:117] "RemoveContainer" containerID="5415d18c45adea897adf687f91a7a2374e5719940e812f6b258bf133a881ed80" Jul 21 06:21:22.621258 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:21:22.621233 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5415d18c45adea897adf687f91a7a2374e5719940e812f6b258bf133a881ed80\": container with ID starting with 5415d18c45adea897adf687f91a7a2374e5719940e812f6b258bf133a881ed80 not found: ID does not exist" containerID="5415d18c45adea897adf687f91a7a2374e5719940e812f6b258bf133a881ed80" Jul 21 06:21:22.621311 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.621270 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5415d18c45adea897adf687f91a7a2374e5719940e812f6b258bf133a881ed80"} err="failed to get container status \"5415d18c45adea897adf687f91a7a2374e5719940e812f6b258bf133a881ed80\": rpc error: code = NotFound desc = could not find container \"5415d18c45adea897adf687f91a7a2374e5719940e812f6b258bf133a881ed80\": container with ID starting with 5415d18c45adea897adf687f91a7a2374e5719940e812f6b258bf133a881ed80 not found: ID does not exist" Jul 21 06:21:22.630739 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.630719 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5465f4d6f8-zhgjj"] Jul 21 06:21:22.634730 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:22.634710 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5465f4d6f8-zhgjj"] Jul 21 06:21:24.485142 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:21:24.485107 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4de1c2c9-85cd-442b-9dab-ebd549d484d3" path="/var/lib/kubelet/pods/4de1c2c9-85cd-442b-9dab-ebd549d484d3/volumes" Jul 21 06:22:00.023358 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.023324 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt"] Jul 21 06:22:00.023770 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.023674 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4de1c2c9-85cd-442b-9dab-ebd549d484d3" containerName="console" Jul 21 06:22:00.023770 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.023689 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4de1c2c9-85cd-442b-9dab-ebd549d484d3" containerName="console" Jul 21 06:22:00.023770 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.023754 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4de1c2c9-85cd-442b-9dab-ebd549d484d3" containerName="console" Jul 21 06:22:00.028126 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.028109 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" Jul 21 06:22:00.031120 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.031099 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 21 06:22:00.031239 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.031165 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sn598\"" Jul 21 06:22:00.031239 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.031099 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 21 06:22:00.036022 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.036002 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt"] Jul 21 06:22:00.148269 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.148233 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47b48675-cfad-44b3-b3a9-ae67834a31ea-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt\" (UID: \"47b48675-cfad-44b3-b3a9-ae67834a31ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" Jul 21 06:22:00.148440 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.148276 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47b48675-cfad-44b3-b3a9-ae67834a31ea-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt\" (UID: \"47b48675-cfad-44b3-b3a9-ae67834a31ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" Jul 21 06:22:00.148440 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.148321 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkbh4\" (UniqueName: \"kubernetes.io/projected/47b48675-cfad-44b3-b3a9-ae67834a31ea-kube-api-access-nkbh4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt\" (UID: \"47b48675-cfad-44b3-b3a9-ae67834a31ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" Jul 21 06:22:00.249180 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.249148 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkbh4\" (UniqueName: \"kubernetes.io/projected/47b48675-cfad-44b3-b3a9-ae67834a31ea-kube-api-access-nkbh4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt\" (UID: \"47b48675-cfad-44b3-b3a9-ae67834a31ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" Jul 21 06:22:00.249333 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.249209 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47b48675-cfad-44b3-b3a9-ae67834a31ea-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt\" (UID: \"47b48675-cfad-44b3-b3a9-ae67834a31ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" Jul 21 06:22:00.249333 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.249249 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47b48675-cfad-44b3-b3a9-ae67834a31ea-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt\" (UID: \"47b48675-cfad-44b3-b3a9-ae67834a31ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" Jul 21 06:22:00.249582 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.249567 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47b48675-cfad-44b3-b3a9-ae67834a31ea-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt\" (UID: \"47b48675-cfad-44b3-b3a9-ae67834a31ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" Jul 21 06:22:00.249620 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.249600 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47b48675-cfad-44b3-b3a9-ae67834a31ea-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt\" (UID: \"47b48675-cfad-44b3-b3a9-ae67834a31ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" Jul 21 06:22:00.260882 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.260855 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkbh4\" (UniqueName: \"kubernetes.io/projected/47b48675-cfad-44b3-b3a9-ae67834a31ea-kube-api-access-nkbh4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt\" (UID: \"47b48675-cfad-44b3-b3a9-ae67834a31ea\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" Jul 21 06:22:00.337974 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.337941 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" Jul 21 06:22:00.468799 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.468744 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt"] Jul 21 06:22:00.470957 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:22:00.470930 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b48675_cfad_44b3_b3a9_ae67834a31ea.slice/crio-b644f5f03627e6fc294f3d163a9dacfe666553ff4b142e88bb5d35f9ab303f92 WatchSource:0}: Error finding container b644f5f03627e6fc294f3d163a9dacfe666553ff4b142e88bb5d35f9ab303f92: Status 404 returned error can't find the container with id b644f5f03627e6fc294f3d163a9dacfe666553ff4b142e88bb5d35f9ab303f92 Jul 21 06:22:00.766068 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:00.765985 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" event={"ID":"47b48675-cfad-44b3-b3a9-ae67834a31ea","Type":"ContainerStarted","Data":"b644f5f03627e6fc294f3d163a9dacfe666553ff4b142e88bb5d35f9ab303f92"} Jul 21 06:22:08.805779 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:08.805742 2563 generic.go:356] "Generic (PLEG): container finished" podID="47b48675-cfad-44b3-b3a9-ae67834a31ea" containerID="ec6165f6b0a79730adbd1378bdf0bfea769850ce543078bc39d0fd6140b3bc03" exitCode=0 Jul 21 06:22:08.806205 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:08.805819 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" event={"ID":"47b48675-cfad-44b3-b3a9-ae67834a31ea","Type":"ContainerDied","Data":"ec6165f6b0a79730adbd1378bdf0bfea769850ce543078bc39d0fd6140b3bc03"} Jul 21 06:22:10.815811 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:10.815783 2563 generic.go:356] "Generic (PLEG): container finished" podID="47b48675-cfad-44b3-b3a9-ae67834a31ea" containerID="2369c14fa9678773b7301140bd1dba65c3c6c25db7a2e379edc9284b58da3d5b" exitCode=0 Jul 21 06:22:10.816192 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:10.815840 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" event={"ID":"47b48675-cfad-44b3-b3a9-ae67834a31ea","Type":"ContainerDied","Data":"2369c14fa9678773b7301140bd1dba65c3c6c25db7a2e379edc9284b58da3d5b"} Jul 21 06:22:18.849547 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:18.849512 2563 generic.go:356] "Generic (PLEG): container finished" podID="47b48675-cfad-44b3-b3a9-ae67834a31ea" containerID="8d3757ee9308dc69005e78abd31695343ff81e20d3ac3b6a1bed1246dca0a41f" exitCode=0 Jul 21 06:22:18.849960 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:18.849575 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" event={"ID":"47b48675-cfad-44b3-b3a9-ae67834a31ea","Type":"ContainerDied","Data":"8d3757ee9308dc69005e78abd31695343ff81e20d3ac3b6a1bed1246dca0a41f"} Jul 21 06:22:19.984239 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:19.984200 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" Jul 21 06:22:20.015164 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:20.015135 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47b48675-cfad-44b3-b3a9-ae67834a31ea-bundle\") pod \"47b48675-cfad-44b3-b3a9-ae67834a31ea\" (UID: \"47b48675-cfad-44b3-b3a9-ae67834a31ea\") " Jul 21 06:22:20.015355 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:20.015237 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkbh4\" (UniqueName: \"kubernetes.io/projected/47b48675-cfad-44b3-b3a9-ae67834a31ea-kube-api-access-nkbh4\") pod \"47b48675-cfad-44b3-b3a9-ae67834a31ea\" (UID: \"47b48675-cfad-44b3-b3a9-ae67834a31ea\") " Jul 21 06:22:20.015355 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:20.015259 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47b48675-cfad-44b3-b3a9-ae67834a31ea-util\") pod \"47b48675-cfad-44b3-b3a9-ae67834a31ea\" (UID: \"47b48675-cfad-44b3-b3a9-ae67834a31ea\") " Jul 21 06:22:20.015827 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:20.015799 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47b48675-cfad-44b3-b3a9-ae67834a31ea-bundle" (OuterVolumeSpecName: "bundle") pod "47b48675-cfad-44b3-b3a9-ae67834a31ea" (UID: "47b48675-cfad-44b3-b3a9-ae67834a31ea"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:22:20.017587 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:20.017568 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47b48675-cfad-44b3-b3a9-ae67834a31ea-kube-api-access-nkbh4" (OuterVolumeSpecName: "kube-api-access-nkbh4") pod "47b48675-cfad-44b3-b3a9-ae67834a31ea" (UID: "47b48675-cfad-44b3-b3a9-ae67834a31ea"). InnerVolumeSpecName "kube-api-access-nkbh4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:22:20.019312 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:20.019266 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47b48675-cfad-44b3-b3a9-ae67834a31ea-util" (OuterVolumeSpecName: "util") pod "47b48675-cfad-44b3-b3a9-ae67834a31ea" (UID: "47b48675-cfad-44b3-b3a9-ae67834a31ea"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:22:20.116154 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:20.116075 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkbh4\" (UniqueName: \"kubernetes.io/projected/47b48675-cfad-44b3-b3a9-ae67834a31ea-kube-api-access-nkbh4\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:22:20.116154 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:20.116103 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47b48675-cfad-44b3-b3a9-ae67834a31ea-util\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:22:20.116154 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:20.116111 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47b48675-cfad-44b3-b3a9-ae67834a31ea-bundle\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:22:20.859091 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:20.859063 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" Jul 21 06:22:20.859272 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:20.859062 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ldfqt" event={"ID":"47b48675-cfad-44b3-b3a9-ae67834a31ea","Type":"ContainerDied","Data":"b644f5f03627e6fc294f3d163a9dacfe666553ff4b142e88bb5d35f9ab303f92"} Jul 21 06:22:20.859272 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:20.859204 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b644f5f03627e6fc294f3d163a9dacfe666553ff4b142e88bb5d35f9ab303f92" Jul 21 06:22:27.990994 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:27.990960 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6"] Jul 21 06:22:27.991393 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:27.991300 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47b48675-cfad-44b3-b3a9-ae67834a31ea" containerName="util" Jul 21 06:22:27.991393 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:27.991311 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="47b48675-cfad-44b3-b3a9-ae67834a31ea" containerName="util" Jul 21 06:22:27.991393 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:27.991320 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47b48675-cfad-44b3-b3a9-ae67834a31ea" containerName="pull" Jul 21 06:22:27.991393 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:27.991325 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="47b48675-cfad-44b3-b3a9-ae67834a31ea" containerName="pull" Jul 21 06:22:27.991393 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:27.991335 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47b48675-cfad-44b3-b3a9-ae67834a31ea" containerName="extract" Jul 21 06:22:27.991393 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:27.991341 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="47b48675-cfad-44b3-b3a9-ae67834a31ea" containerName="extract" Jul 21 06:22:27.991393 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:27.991390 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="47b48675-cfad-44b3-b3a9-ae67834a31ea" containerName="extract" Jul 21 06:22:28.025515 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.025469 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6"] Jul 21 06:22:28.025695 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.025574 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6" Jul 21 06:22:28.029121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.029084 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-gkj7n\"" Jul 21 06:22:28.029312 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.029244 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 21 06:22:28.029423 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.029405 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 21 06:22:28.076320 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.076283 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfsqp\" (UniqueName: \"kubernetes.io/projected/18a569b4-6834-46b3-ab37-a4618b5f23bd-kube-api-access-nfsqp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rfdc6\" (UID: \"18a569b4-6834-46b3-ab37-a4618b5f23bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6" Jul 21 06:22:28.076493 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.076334 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18a569b4-6834-46b3-ab37-a4618b5f23bd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rfdc6\" (UID: \"18a569b4-6834-46b3-ab37-a4618b5f23bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6" Jul 21 06:22:28.177551 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.177521 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfsqp\" (UniqueName: \"kubernetes.io/projected/18a569b4-6834-46b3-ab37-a4618b5f23bd-kube-api-access-nfsqp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rfdc6\" (UID: \"18a569b4-6834-46b3-ab37-a4618b5f23bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6" Jul 21 06:22:28.177723 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.177562 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18a569b4-6834-46b3-ab37-a4618b5f23bd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rfdc6\" (UID: \"18a569b4-6834-46b3-ab37-a4618b5f23bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6" Jul 21 06:22:28.177919 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.177900 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18a569b4-6834-46b3-ab37-a4618b5f23bd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rfdc6\" (UID: \"18a569b4-6834-46b3-ab37-a4618b5f23bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6" Jul 21 06:22:28.188985 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.188954 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfsqp\" (UniqueName: \"kubernetes.io/projected/18a569b4-6834-46b3-ab37-a4618b5f23bd-kube-api-access-nfsqp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rfdc6\" (UID: \"18a569b4-6834-46b3-ab37-a4618b5f23bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6" Jul 21 06:22:28.338079 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.338040 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6" Jul 21 06:22:28.492899 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.492825 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6"] Jul 21 06:22:28.495288 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:22:28.495264 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a569b4_6834_46b3_ab37_a4618b5f23bd.slice/crio-8149cac041c72e9971eae1c508d7a2857d4ea775f1fc4631d637874405137804 WatchSource:0}: Error finding container 8149cac041c72e9971eae1c508d7a2857d4ea775f1fc4631d637874405137804: Status 404 returned error can't find the container with id 8149cac041c72e9971eae1c508d7a2857d4ea775f1fc4631d637874405137804 Jul 21 06:22:28.892920 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:28.892887 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6" event={"ID":"18a569b4-6834-46b3-ab37-a4618b5f23bd","Type":"ContainerStarted","Data":"8149cac041c72e9971eae1c508d7a2857d4ea775f1fc4631d637874405137804"} Jul 21 06:22:30.904862 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:30.904829 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6" event={"ID":"18a569b4-6834-46b3-ab37-a4618b5f23bd","Type":"ContainerStarted","Data":"c03f5c30c51016ff7e5436c69cbd73f7d98ef0078d3fe1845ca2153bf9f204e5"} Jul 21 06:22:30.926634 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:30.926584 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rfdc6" podStartSLOduration=1.726997154 podStartE2EDuration="3.926564884s" totalImagesPullingTime="2.19956773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:22:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:22:30.924667156 +0000 UTC m=+280.870305816" watchObservedRunningTime="2026-07-21 06:22:30.926564884 +0000 UTC m=+280.872203542" Jul 21 06:22:32.058513 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.058478 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2"] Jul 21 06:22:32.060798 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.060782 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" Jul 21 06:22:32.063815 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.063787 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 21 06:22:32.063938 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.063886 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 21 06:22:32.065381 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.065366 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sn598\"" Jul 21 06:22:32.077341 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.077317 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2"] Jul 21 06:22:32.112371 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.112340 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd24d431-f5c4-4d46-97ff-1f3e2508873b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2\" (UID: \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" Jul 21 06:22:32.112371 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.112372 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd24d431-f5c4-4d46-97ff-1f3e2508873b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2\" (UID: \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" Jul 21 06:22:32.112554 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.112474 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4fzq\" (UniqueName: \"kubernetes.io/projected/cd24d431-f5c4-4d46-97ff-1f3e2508873b-kube-api-access-d4fzq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2\" (UID: \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" Jul 21 06:22:32.213577 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.213544 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4fzq\" (UniqueName: \"kubernetes.io/projected/cd24d431-f5c4-4d46-97ff-1f3e2508873b-kube-api-access-d4fzq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2\" (UID: \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" Jul 21 06:22:32.213766 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.213612 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd24d431-f5c4-4d46-97ff-1f3e2508873b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2\" (UID: \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" Jul 21 06:22:32.213766 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.213640 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd24d431-f5c4-4d46-97ff-1f3e2508873b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2\" (UID: \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" Jul 21 06:22:32.213979 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.213958 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd24d431-f5c4-4d46-97ff-1f3e2508873b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2\" (UID: \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" Jul 21 06:22:32.214060 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.214005 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd24d431-f5c4-4d46-97ff-1f3e2508873b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2\" (UID: \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" Jul 21 06:22:32.224968 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.224941 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4fzq\" (UniqueName: \"kubernetes.io/projected/cd24d431-f5c4-4d46-97ff-1f3e2508873b-kube-api-access-d4fzq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2\" (UID: \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" Jul 21 06:22:32.370321 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.370290 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" Jul 21 06:22:32.494636 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.494611 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2"] Jul 21 06:22:32.496661 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:22:32.496637 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd24d431_f5c4_4d46_97ff_1f3e2508873b.slice/crio-69edaae6cfb0a39222b30ef496eeb04853512b3a357483dcef2b55e08fc49218 WatchSource:0}: Error finding container 69edaae6cfb0a39222b30ef496eeb04853512b3a357483dcef2b55e08fc49218: Status 404 returned error can't find the container with id 69edaae6cfb0a39222b30ef496eeb04853512b3a357483dcef2b55e08fc49218 Jul 21 06:22:32.914648 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.914612 2563 generic.go:356] "Generic (PLEG): container finished" podID="cd24d431-f5c4-4d46-97ff-1f3e2508873b" containerID="f474000936b888fa7b5477bd56ac53d26ab8cb1a9836d97bde0c87bd88eba16e" exitCode=0 Jul 21 06:22:32.914810 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.914697 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" event={"ID":"cd24d431-f5c4-4d46-97ff-1f3e2508873b","Type":"ContainerDied","Data":"f474000936b888fa7b5477bd56ac53d26ab8cb1a9836d97bde0c87bd88eba16e"} Jul 21 06:22:32.914810 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:32.914751 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" event={"ID":"cd24d431-f5c4-4d46-97ff-1f3e2508873b","Type":"ContainerStarted","Data":"69edaae6cfb0a39222b30ef496eeb04853512b3a357483dcef2b55e08fc49218"} Jul 21 06:22:34.197834 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.197797 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g"] Jul 21 06:22:34.203966 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.203946 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g" Jul 21 06:22:34.207034 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.206756 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 21 06:22:34.207034 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.206870 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 21 06:22:34.207034 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.206869 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-76wdb\"" Jul 21 06:22:34.207887 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.207864 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g"] Jul 21 06:22:34.230313 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.230272 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hbcv\" (UniqueName: \"kubernetes.io/projected/f1063403-7679-4d40-bbbd-c5530147a89f-kube-api-access-5hbcv\") pod \"cert-manager-cainjector-559cdd8c87-6rr6g\" (UID: \"f1063403-7679-4d40-bbbd-c5530147a89f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g" Jul 21 06:22:34.230423 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.230357 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1063403-7679-4d40-bbbd-c5530147a89f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6rr6g\" (UID: \"f1063403-7679-4d40-bbbd-c5530147a89f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g" Jul 21 06:22:34.331343 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.331314 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hbcv\" (UniqueName: \"kubernetes.io/projected/f1063403-7679-4d40-bbbd-c5530147a89f-kube-api-access-5hbcv\") pod \"cert-manager-cainjector-559cdd8c87-6rr6g\" (UID: \"f1063403-7679-4d40-bbbd-c5530147a89f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g" Jul 21 06:22:34.331514 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.331380 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1063403-7679-4d40-bbbd-c5530147a89f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6rr6g\" (UID: \"f1063403-7679-4d40-bbbd-c5530147a89f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g" Jul 21 06:22:34.339767 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.339740 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1063403-7679-4d40-bbbd-c5530147a89f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6rr6g\" (UID: \"f1063403-7679-4d40-bbbd-c5530147a89f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g" Jul 21 06:22:34.339767 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.339765 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hbcv\" (UniqueName: \"kubernetes.io/projected/f1063403-7679-4d40-bbbd-c5530147a89f-kube-api-access-5hbcv\") pod \"cert-manager-cainjector-559cdd8c87-6rr6g\" (UID: \"f1063403-7679-4d40-bbbd-c5530147a89f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g" Jul 21 06:22:34.533607 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.533530 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g" Jul 21 06:22:34.798248 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.798207 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g"] Jul 21 06:22:34.800566 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:22:34.800538 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1063403_7679_4d40_bbbd_c5530147a89f.slice/crio-35de1c9ec3c19189766f25546b9623a243283f87fc858c715c782c454707aece WatchSource:0}: Error finding container 35de1c9ec3c19189766f25546b9623a243283f87fc858c715c782c454707aece: Status 404 returned error can't find the container with id 35de1c9ec3c19189766f25546b9623a243283f87fc858c715c782c454707aece Jul 21 06:22:34.924087 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.924054 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g" event={"ID":"f1063403-7679-4d40-bbbd-c5530147a89f","Type":"ContainerStarted","Data":"35de1c9ec3c19189766f25546b9623a243283f87fc858c715c782c454707aece"} Jul 21 06:22:34.925759 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.925739 2563 generic.go:356] "Generic (PLEG): container finished" podID="cd24d431-f5c4-4d46-97ff-1f3e2508873b" containerID="2b052d85d45eee0b3453baebde4a75eddc89d9d63e7e4a174231bff1dc3fc76a" exitCode=0 Jul 21 06:22:34.925873 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:34.925794 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" event={"ID":"cd24d431-f5c4-4d46-97ff-1f3e2508873b","Type":"ContainerDied","Data":"2b052d85d45eee0b3453baebde4a75eddc89d9d63e7e4a174231bff1dc3fc76a"} Jul 21 06:22:35.933816 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:35.933780 2563 generic.go:356] "Generic (PLEG): container finished" podID="cd24d431-f5c4-4d46-97ff-1f3e2508873b" containerID="a5169facfdfe25595119c1d62ee1156e5380c546757cc225ac5db874b014f24f" exitCode=0 Jul 21 06:22:35.934257 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:35.933865 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" event={"ID":"cd24d431-f5c4-4d46-97ff-1f3e2508873b","Type":"ContainerDied","Data":"a5169facfdfe25595119c1d62ee1156e5380c546757cc225ac5db874b014f24f"} Jul 21 06:22:37.247283 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.247255 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" Jul 21 06:22:37.359841 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.359803 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd24d431-f5c4-4d46-97ff-1f3e2508873b-bundle\") pod \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\" (UID: \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\") " Jul 21 06:22:37.360004 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.359861 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd24d431-f5c4-4d46-97ff-1f3e2508873b-util\") pod \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\" (UID: \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\") " Jul 21 06:22:37.360004 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.359890 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4fzq\" (UniqueName: \"kubernetes.io/projected/cd24d431-f5c4-4d46-97ff-1f3e2508873b-kube-api-access-d4fzq\") pod \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\" (UID: \"cd24d431-f5c4-4d46-97ff-1f3e2508873b\") " Jul 21 06:22:37.360271 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.360238 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd24d431-f5c4-4d46-97ff-1f3e2508873b-bundle" (OuterVolumeSpecName: "bundle") pod "cd24d431-f5c4-4d46-97ff-1f3e2508873b" (UID: "cd24d431-f5c4-4d46-97ff-1f3e2508873b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:22:37.362392 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.362361 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd24d431-f5c4-4d46-97ff-1f3e2508873b-kube-api-access-d4fzq" (OuterVolumeSpecName: "kube-api-access-d4fzq") pod "cd24d431-f5c4-4d46-97ff-1f3e2508873b" (UID: "cd24d431-f5c4-4d46-97ff-1f3e2508873b"). InnerVolumeSpecName "kube-api-access-d4fzq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:22:37.365498 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.365474 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd24d431-f5c4-4d46-97ff-1f3e2508873b-util" (OuterVolumeSpecName: "util") pod "cd24d431-f5c4-4d46-97ff-1f3e2508873b" (UID: "cd24d431-f5c4-4d46-97ff-1f3e2508873b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:22:37.460661 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.460625 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd24d431-f5c4-4d46-97ff-1f3e2508873b-bundle\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:22:37.460661 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.460654 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd24d431-f5c4-4d46-97ff-1f3e2508873b-util\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:22:37.460661 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.460663 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4fzq\" (UniqueName: \"kubernetes.io/projected/cd24d431-f5c4-4d46-97ff-1f3e2508873b-kube-api-access-d4fzq\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:22:37.943425 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.943391 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g" event={"ID":"f1063403-7679-4d40-bbbd-c5530147a89f","Type":"ContainerStarted","Data":"1cbd2d8a17db99db7383da2ece1e53da6146394e42922313294a552666d67048"} Jul 21 06:22:37.945163 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.945143 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" Jul 21 06:22:37.945297 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.945158 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxznd2" event={"ID":"cd24d431-f5c4-4d46-97ff-1f3e2508873b","Type":"ContainerDied","Data":"69edaae6cfb0a39222b30ef496eeb04853512b3a357483dcef2b55e08fc49218"} Jul 21 06:22:37.945297 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.945195 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69edaae6cfb0a39222b30ef496eeb04853512b3a357483dcef2b55e08fc49218" Jul 21 06:22:37.958279 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:37.958203 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6rr6g" podStartSLOduration=1.492806984 podStartE2EDuration="3.958189156s" totalImagesPullingTime="2.465382172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:22:37.957524102 +0000 UTC m=+287.903162772" watchObservedRunningTime="2026-07-21 06:22:37.958189156 +0000 UTC m=+287.903827813" Jul 21 06:22:38.382299 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.382260 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xf5pk"] Jul 21 06:22:38.382664 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.382569 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd24d431-f5c4-4d46-97ff-1f3e2508873b" containerName="extract" Jul 21 06:22:38.382664 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.382579 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd24d431-f5c4-4d46-97ff-1f3e2508873b" containerName="extract" Jul 21 06:22:38.382664 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.382590 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd24d431-f5c4-4d46-97ff-1f3e2508873b" containerName="util" Jul 21 06:22:38.382664 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.382595 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd24d431-f5c4-4d46-97ff-1f3e2508873b" containerName="util" Jul 21 06:22:38.382664 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.382604 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd24d431-f5c4-4d46-97ff-1f3e2508873b" containerName="pull" Jul 21 06:22:38.382664 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.382610 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd24d431-f5c4-4d46-97ff-1f3e2508873b" containerName="pull" Jul 21 06:22:38.382834 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.382683 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd24d431-f5c4-4d46-97ff-1f3e2508873b" containerName="extract" Jul 21 06:22:38.384710 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.384694 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xf5pk" Jul 21 06:22:38.387393 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.387374 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-vf8vb\"" Jul 21 06:22:38.395452 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.395430 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xf5pk"] Jul 21 06:22:38.468887 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.468847 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c20df43-3f32-49a5-87af-ecf9d9faff16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xf5pk\" (UID: \"6c20df43-3f32-49a5-87af-ecf9d9faff16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xf5pk" Jul 21 06:22:38.469067 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.468905 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjncq\" (UniqueName: \"kubernetes.io/projected/6c20df43-3f32-49a5-87af-ecf9d9faff16-kube-api-access-tjncq\") pod \"cert-manager-webhook-6486ffd997-xf5pk\" (UID: \"6c20df43-3f32-49a5-87af-ecf9d9faff16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xf5pk" Jul 21 06:22:38.569923 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.569886 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjncq\" (UniqueName: \"kubernetes.io/projected/6c20df43-3f32-49a5-87af-ecf9d9faff16-kube-api-access-tjncq\") pod \"cert-manager-webhook-6486ffd997-xf5pk\" (UID: \"6c20df43-3f32-49a5-87af-ecf9d9faff16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xf5pk" Jul 21 06:22:38.570127 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.570024 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c20df43-3f32-49a5-87af-ecf9d9faff16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xf5pk\" (UID: \"6c20df43-3f32-49a5-87af-ecf9d9faff16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xf5pk" Jul 21 06:22:38.585117 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.585088 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6c20df43-3f32-49a5-87af-ecf9d9faff16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xf5pk\" (UID: \"6c20df43-3f32-49a5-87af-ecf9d9faff16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xf5pk" Jul 21 06:22:38.585324 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.585300 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjncq\" (UniqueName: \"kubernetes.io/projected/6c20df43-3f32-49a5-87af-ecf9d9faff16-kube-api-access-tjncq\") pod \"cert-manager-webhook-6486ffd997-xf5pk\" (UID: \"6c20df43-3f32-49a5-87af-ecf9d9faff16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xf5pk" Jul 21 06:22:38.694027 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.693941 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xf5pk" Jul 21 06:22:38.822484 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.822409 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xf5pk"] Jul 21 06:22:38.824620 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:22:38.824595 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c20df43_3f32_49a5_87af_ecf9d9faff16.slice/crio-9d07e623891476878995be504965108fc15dfad3b70f79f60f3d150286435409 WatchSource:0}: Error finding container 9d07e623891476878995be504965108fc15dfad3b70f79f60f3d150286435409: Status 404 returned error can't find the container with id 9d07e623891476878995be504965108fc15dfad3b70f79f60f3d150286435409 Jul 21 06:22:38.951534 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.951450 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xf5pk" event={"ID":"6c20df43-3f32-49a5-87af-ecf9d9faff16","Type":"ContainerStarted","Data":"3fb3078b738ba78cdcb4cb4f1b68c68749f0f4e773eb0ae05b9e5b91db8ad153"} Jul 21 06:22:38.951534 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.951486 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xf5pk" event={"ID":"6c20df43-3f32-49a5-87af-ecf9d9faff16","Type":"ContainerStarted","Data":"9d07e623891476878995be504965108fc15dfad3b70f79f60f3d150286435409"} Jul 21 06:22:38.951720 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.951557 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xf5pk" Jul 21 06:22:38.968340 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:38.968294 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-xf5pk" podStartSLOduration=0.968277592 podStartE2EDuration="968.277592ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:22:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:22:38.966258131 +0000 UTC m=+288.911896787" watchObservedRunningTime="2026-07-21 06:22:38.968277592 +0000 UTC m=+288.913916250" Jul 21 06:22:44.252389 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.252356 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9qkjr"] Jul 21 06:22:44.254435 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.254419 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9qkjr" Jul 21 06:22:44.257365 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.257350 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-jz2nb\"" Jul 21 06:22:44.264832 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.264812 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9qkjr"] Jul 21 06:22:44.322580 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.322549 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0505a792-6d54-49b6-8b10-0c647d51c6e5-bound-sa-token\") pod \"cert-manager-55b4d954d5-9qkjr\" (UID: \"0505a792-6d54-49b6-8b10-0c647d51c6e5\") " pod="cert-manager/cert-manager-55b4d954d5-9qkjr" Jul 21 06:22:44.322729 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.322599 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs974\" (UniqueName: \"kubernetes.io/projected/0505a792-6d54-49b6-8b10-0c647d51c6e5-kube-api-access-qs974\") pod \"cert-manager-55b4d954d5-9qkjr\" (UID: \"0505a792-6d54-49b6-8b10-0c647d51c6e5\") " pod="cert-manager/cert-manager-55b4d954d5-9qkjr" Jul 21 06:22:44.423941 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.423912 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs974\" (UniqueName: \"kubernetes.io/projected/0505a792-6d54-49b6-8b10-0c647d51c6e5-kube-api-access-qs974\") pod \"cert-manager-55b4d954d5-9qkjr\" (UID: \"0505a792-6d54-49b6-8b10-0c647d51c6e5\") " pod="cert-manager/cert-manager-55b4d954d5-9qkjr" Jul 21 06:22:44.424062 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.423987 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0505a792-6d54-49b6-8b10-0c647d51c6e5-bound-sa-token\") pod \"cert-manager-55b4d954d5-9qkjr\" (UID: \"0505a792-6d54-49b6-8b10-0c647d51c6e5\") " pod="cert-manager/cert-manager-55b4d954d5-9qkjr" Jul 21 06:22:44.432367 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.432341 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0505a792-6d54-49b6-8b10-0c647d51c6e5-bound-sa-token\") pod \"cert-manager-55b4d954d5-9qkjr\" (UID: \"0505a792-6d54-49b6-8b10-0c647d51c6e5\") " pod="cert-manager/cert-manager-55b4d954d5-9qkjr" Jul 21 06:22:44.432475 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.432401 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs974\" (UniqueName: \"kubernetes.io/projected/0505a792-6d54-49b6-8b10-0c647d51c6e5-kube-api-access-qs974\") pod \"cert-manager-55b4d954d5-9qkjr\" (UID: \"0505a792-6d54-49b6-8b10-0c647d51c6e5\") " pod="cert-manager/cert-manager-55b4d954d5-9qkjr" Jul 21 06:22:44.563744 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.563669 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9qkjr" Jul 21 06:22:44.689021 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.688997 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9qkjr"] Jul 21 06:22:44.691247 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:22:44.691202 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0505a792_6d54_49b6_8b10_0c647d51c6e5.slice/crio-628caa012a02df16c4c8c47461d81c18240ffe076e76f47450cc52f2b67a479e WatchSource:0}: Error finding container 628caa012a02df16c4c8c47461d81c18240ffe076e76f47450cc52f2b67a479e: Status 404 returned error can't find the container with id 628caa012a02df16c4c8c47461d81c18240ffe076e76f47450cc52f2b67a479e Jul 21 06:22:44.958416 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.958390 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xf5pk" Jul 21 06:22:44.978575 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.978543 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9qkjr" event={"ID":"0505a792-6d54-49b6-8b10-0c647d51c6e5","Type":"ContainerStarted","Data":"e5c31587e08d7691d38966c7a697fe290539e332d32de688e2ed36f70b09e5ee"} Jul 21 06:22:44.978575 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.978578 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9qkjr" event={"ID":"0505a792-6d54-49b6-8b10-0c647d51c6e5","Type":"ContainerStarted","Data":"628caa012a02df16c4c8c47461d81c18240ffe076e76f47450cc52f2b67a479e"} Jul 21 06:22:44.996985 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:44.996937 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9qkjr" podStartSLOduration=0.996921571 podStartE2EDuration="996.921571ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:22:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:22:44.995113952 +0000 UTC m=+294.940752614" watchObservedRunningTime="2026-07-21 06:22:44.996921571 +0000 UTC m=+294.942560230" Jul 21 06:22:48.100143 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.100111 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb"] Jul 21 06:22:48.102488 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.102472 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" Jul 21 06:22:48.105386 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.105365 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sn598\"" Jul 21 06:22:48.105490 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.105424 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 21 06:22:48.106855 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.106835 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 21 06:22:48.115630 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.115599 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb"] Jul 21 06:22:48.153819 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.153778 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a77e0b7b-a187-486c-9fbe-5df767404b96-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb\" (UID: \"a77e0b7b-a187-486c-9fbe-5df767404b96\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" Jul 21 06:22:48.153819 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.153823 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a77e0b7b-a187-486c-9fbe-5df767404b96-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb\" (UID: \"a77e0b7b-a187-486c-9fbe-5df767404b96\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" Jul 21 06:22:48.154026 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.153841 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rgjf\" (UniqueName: \"kubernetes.io/projected/a77e0b7b-a187-486c-9fbe-5df767404b96-kube-api-access-2rgjf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb\" (UID: \"a77e0b7b-a187-486c-9fbe-5df767404b96\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" Jul 21 06:22:48.255083 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.255038 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a77e0b7b-a187-486c-9fbe-5df767404b96-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb\" (UID: \"a77e0b7b-a187-486c-9fbe-5df767404b96\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" Jul 21 06:22:48.255083 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.255082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a77e0b7b-a187-486c-9fbe-5df767404b96-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb\" (UID: \"a77e0b7b-a187-486c-9fbe-5df767404b96\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" Jul 21 06:22:48.255306 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.255100 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rgjf\" (UniqueName: \"kubernetes.io/projected/a77e0b7b-a187-486c-9fbe-5df767404b96-kube-api-access-2rgjf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb\" (UID: \"a77e0b7b-a187-486c-9fbe-5df767404b96\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" Jul 21 06:22:48.255564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.255538 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a77e0b7b-a187-486c-9fbe-5df767404b96-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb\" (UID: \"a77e0b7b-a187-486c-9fbe-5df767404b96\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" Jul 21 06:22:48.255606 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.255546 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a77e0b7b-a187-486c-9fbe-5df767404b96-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb\" (UID: \"a77e0b7b-a187-486c-9fbe-5df767404b96\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" Jul 21 06:22:48.263245 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.263193 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rgjf\" (UniqueName: \"kubernetes.io/projected/a77e0b7b-a187-486c-9fbe-5df767404b96-kube-api-access-2rgjf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb\" (UID: \"a77e0b7b-a187-486c-9fbe-5df767404b96\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" Jul 21 06:22:48.412473 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.412390 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" Jul 21 06:22:48.543456 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.543420 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb"] Jul 21 06:22:48.545625 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:22:48.545598 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda77e0b7b_a187_486c_9fbe_5df767404b96.slice/crio-ae2e516c885d224fec8b79d97b1c89c60f1c4a589eb41aab4c8ed0d0e79673d3 WatchSource:0}: Error finding container ae2e516c885d224fec8b79d97b1c89c60f1c4a589eb41aab4c8ed0d0e79673d3: Status 404 returned error can't find the container with id ae2e516c885d224fec8b79d97b1c89c60f1c4a589eb41aab4c8ed0d0e79673d3 Jul 21 06:22:48.997719 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.997686 2563 generic.go:356] "Generic (PLEG): container finished" podID="a77e0b7b-a187-486c-9fbe-5df767404b96" containerID="368f7ee33407feb023243d485316cf94c9ed3bad24d1d05d124f0e6c5d18160b" exitCode=0 Jul 21 06:22:48.997915 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.997790 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" event={"ID":"a77e0b7b-a187-486c-9fbe-5df767404b96","Type":"ContainerDied","Data":"368f7ee33407feb023243d485316cf94c9ed3bad24d1d05d124f0e6c5d18160b"} Jul 21 06:22:48.997915 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:48.997850 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" event={"ID":"a77e0b7b-a187-486c-9fbe-5df767404b96","Type":"ContainerStarted","Data":"ae2e516c885d224fec8b79d97b1c89c60f1c4a589eb41aab4c8ed0d0e79673d3"} Jul 21 06:22:50.003696 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:50.003601 2563 generic.go:356] "Generic (PLEG): container finished" podID="a77e0b7b-a187-486c-9fbe-5df767404b96" containerID="f14fe7bffe7aaddec79ad5ac54b20abb0be610482dd3e17e39cafd31c3f05ad4" exitCode=0 Jul 21 06:22:50.004057 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:50.003688 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" event={"ID":"a77e0b7b-a187-486c-9fbe-5df767404b96","Type":"ContainerDied","Data":"f14fe7bffe7aaddec79ad5ac54b20abb0be610482dd3e17e39cafd31c3f05ad4"} Jul 21 06:22:50.401357 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:50.401336 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 06:22:51.010123 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:51.010094 2563 generic.go:356] "Generic (PLEG): container finished" podID="a77e0b7b-a187-486c-9fbe-5df767404b96" containerID="fc2bd9477fa016485932fbb21ce897c21ad50aef5c24146b399206ffaafbf872" exitCode=0 Jul 21 06:22:51.011365 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:51.010181 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" event={"ID":"a77e0b7b-a187-486c-9fbe-5df767404b96","Type":"ContainerDied","Data":"fc2bd9477fa016485932fbb21ce897c21ad50aef5c24146b399206ffaafbf872"} Jul 21 06:22:52.138568 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:52.138545 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" Jul 21 06:22:52.288330 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:52.288247 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rgjf\" (UniqueName: \"kubernetes.io/projected/a77e0b7b-a187-486c-9fbe-5df767404b96-kube-api-access-2rgjf\") pod \"a77e0b7b-a187-486c-9fbe-5df767404b96\" (UID: \"a77e0b7b-a187-486c-9fbe-5df767404b96\") " Jul 21 06:22:52.288330 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:52.288298 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a77e0b7b-a187-486c-9fbe-5df767404b96-util\") pod \"a77e0b7b-a187-486c-9fbe-5df767404b96\" (UID: \"a77e0b7b-a187-486c-9fbe-5df767404b96\") " Jul 21 06:22:52.288524 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:52.288428 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a77e0b7b-a187-486c-9fbe-5df767404b96-bundle\") pod \"a77e0b7b-a187-486c-9fbe-5df767404b96\" (UID: \"a77e0b7b-a187-486c-9fbe-5df767404b96\") " Jul 21 06:22:52.289096 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:52.289064 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a77e0b7b-a187-486c-9fbe-5df767404b96-bundle" (OuterVolumeSpecName: "bundle") pod "a77e0b7b-a187-486c-9fbe-5df767404b96" (UID: "a77e0b7b-a187-486c-9fbe-5df767404b96"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:22:52.290615 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:52.290595 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a77e0b7b-a187-486c-9fbe-5df767404b96-kube-api-access-2rgjf" (OuterVolumeSpecName: "kube-api-access-2rgjf") pod "a77e0b7b-a187-486c-9fbe-5df767404b96" (UID: "a77e0b7b-a187-486c-9fbe-5df767404b96"). InnerVolumeSpecName "kube-api-access-2rgjf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:22:52.293821 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:52.293802 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a77e0b7b-a187-486c-9fbe-5df767404b96-util" (OuterVolumeSpecName: "util") pod "a77e0b7b-a187-486c-9fbe-5df767404b96" (UID: "a77e0b7b-a187-486c-9fbe-5df767404b96"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:22:52.389519 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:52.389485 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a77e0b7b-a187-486c-9fbe-5df767404b96-util\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:22:52.389519 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:52.389513 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a77e0b7b-a187-486c-9fbe-5df767404b96-bundle\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:22:52.389519 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:52.389522 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rgjf\" (UniqueName: \"kubernetes.io/projected/a77e0b7b-a187-486c-9fbe-5df767404b96-kube-api-access-2rgjf\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:22:53.021229 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:53.021186 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" event={"ID":"a77e0b7b-a187-486c-9fbe-5df767404b96","Type":"ContainerDied","Data":"ae2e516c885d224fec8b79d97b1c89c60f1c4a589eb41aab4c8ed0d0e79673d3"} Jul 21 06:22:53.021393 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:53.021254 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbmzbb" Jul 21 06:22:53.021393 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:22:53.021267 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae2e516c885d224fec8b79d97b1c89c60f1c4a589eb41aab4c8ed0d0e79673d3" Jul 21 06:23:04.381678 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.381642 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q"] Jul 21 06:23:04.382069 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.381979 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a77e0b7b-a187-486c-9fbe-5df767404b96" containerName="pull" Jul 21 06:23:04.382069 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.381990 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a77e0b7b-a187-486c-9fbe-5df767404b96" containerName="pull" Jul 21 06:23:04.382069 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.382006 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a77e0b7b-a187-486c-9fbe-5df767404b96" containerName="util" Jul 21 06:23:04.382069 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.382011 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a77e0b7b-a187-486c-9fbe-5df767404b96" containerName="util" Jul 21 06:23:04.382069 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.382027 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a77e0b7b-a187-486c-9fbe-5df767404b96" containerName="extract" Jul 21 06:23:04.382069 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.382032 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a77e0b7b-a187-486c-9fbe-5df767404b96" containerName="extract" Jul 21 06:23:04.382281 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.382080 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a77e0b7b-a187-486c-9fbe-5df767404b96" containerName="extract" Jul 21 06:23:04.388062 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.388043 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.391060 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.391042 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 06:23:04.391306 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.391291 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 06:23:04.391388 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.391348 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7pnhr\"" Jul 21 06:23:04.391388 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.391294 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 06:23:04.391489 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.391296 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 06:23:04.398821 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.398796 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q"] Jul 21 06:23:04.493833 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.493801 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b494fd0f-dff8-4843-8e75-2b326e9b2323-webhook-cert\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.493833 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.493834 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b494fd0f-dff8-4843-8e75-2b326e9b2323-tmp\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.494014 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.493864 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b494fd0f-dff8-4843-8e75-2b326e9b2323-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.494014 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.493951 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbl5d\" (UniqueName: \"kubernetes.io/projected/b494fd0f-dff8-4843-8e75-2b326e9b2323-kube-api-access-jbl5d\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.494014 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.493986 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b494fd0f-dff8-4843-8e75-2b326e9b2323-manifests\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.594904 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.594875 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b494fd0f-dff8-4843-8e75-2b326e9b2323-webhook-cert\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.594904 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.594908 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b494fd0f-dff8-4843-8e75-2b326e9b2323-tmp\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.595090 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.594942 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b494fd0f-dff8-4843-8e75-2b326e9b2323-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.595090 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.594984 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbl5d\" (UniqueName: \"kubernetes.io/projected/b494fd0f-dff8-4843-8e75-2b326e9b2323-kube-api-access-jbl5d\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.595090 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.595010 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b494fd0f-dff8-4843-8e75-2b326e9b2323-manifests\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.595450 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.595421 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b494fd0f-dff8-4843-8e75-2b326e9b2323-manifests\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.597386 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.597358 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b494fd0f-dff8-4843-8e75-2b326e9b2323-tmp\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.597495 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.597481 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b494fd0f-dff8-4843-8e75-2b326e9b2323-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.597568 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.597552 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b494fd0f-dff8-4843-8e75-2b326e9b2323-webhook-cert\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.608325 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.608305 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbl5d\" (UniqueName: \"kubernetes.io/projected/b494fd0f-dff8-4843-8e75-2b326e9b2323-kube-api-access-jbl5d\") pod \"opendatahub-operator-controller-manager-78c844c949-9v75q\" (UID: \"b494fd0f-dff8-4843-8e75-2b326e9b2323\") " pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.699542 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.699460 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:04.836148 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.836125 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q"] Jul 21 06:23:04.838102 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:23:04.838078 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb494fd0f_dff8_4843_8e75_2b326e9b2323.slice/crio-ef2376d174a14a423e5c306ab8b13b90626dfc544a30ef39b091796ba7034b3f WatchSource:0}: Error finding container ef2376d174a14a423e5c306ab8b13b90626dfc544a30ef39b091796ba7034b3f: Status 404 returned error can't find the container with id ef2376d174a14a423e5c306ab8b13b90626dfc544a30ef39b091796ba7034b3f Jul 21 06:23:04.839844 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:04.839827 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 06:23:05.077433 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:05.077402 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" event={"ID":"b494fd0f-dff8-4843-8e75-2b326e9b2323","Type":"ContainerStarted","Data":"ef2376d174a14a423e5c306ab8b13b90626dfc544a30ef39b091796ba7034b3f"} Jul 21 06:23:08.099766 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:08.099731 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" event={"ID":"b494fd0f-dff8-4843-8e75-2b326e9b2323","Type":"ContainerStarted","Data":"8209b1a5ffb1a044476b9353718ac7cecf5e73c5052305bc10f8b6bfbc0c8425"} Jul 21 06:23:09.106733 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:09.106697 2563 generic.go:356] "Generic (PLEG): container finished" podID="b494fd0f-dff8-4843-8e75-2b326e9b2323" containerID="8209b1a5ffb1a044476b9353718ac7cecf5e73c5052305bc10f8b6bfbc0c8425" exitCode=0 Jul 21 06:23:09.107117 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:09.106795 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" event={"ID":"b494fd0f-dff8-4843-8e75-2b326e9b2323","Type":"ContainerDied","Data":"8209b1a5ffb1a044476b9353718ac7cecf5e73c5052305bc10f8b6bfbc0c8425"} Jul 21 06:23:10.114387 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:10.114354 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" event={"ID":"b494fd0f-dff8-4843-8e75-2b326e9b2323","Type":"ContainerStarted","Data":"2c476d9ac330d88c05960e0b61eafa031300f3c779b68755dbc0655377b32308"} Jul 21 06:23:10.114768 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:10.114498 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:10.140665 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:10.140621 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" podStartSLOduration=1.6577643100000001 podStartE2EDuration="6.140606818s" totalImagesPullingTime="3.472933107s" totalInitContainerRuntime="1.009909401s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:23:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:23:10.139586147 +0000 UTC m=+320.085224806" watchObservedRunningTime="2026-07-21 06:23:10.140606818 +0000 UTC m=+320.086245477" Jul 21 06:23:21.122042 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:21.122007 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78c844c949-9v75q" Jul 21 06:23:23.451438 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.451405 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6"] Jul 21 06:23:23.458591 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.458564 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" Jul 21 06:23:23.461587 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.461563 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 21 06:23:23.461748 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.461666 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2sfdx\"" Jul 21 06:23:23.463154 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.463131 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 21 06:23:23.463303 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.463178 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 06:23:23.463303 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.463261 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 06:23:23.476883 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.476858 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6"] Jul 21 06:23:23.555070 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.555030 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5418a32d-6c58-4113-8ce4-52753a3d4401-tmp\") pod \"kube-auth-proxy-5ff9c87565-srhn6\" (UID: \"5418a32d-6c58-4113-8ce4-52753a3d4401\") " pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" Jul 21 06:23:23.555070 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.555075 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qznst\" (UniqueName: \"kubernetes.io/projected/5418a32d-6c58-4113-8ce4-52753a3d4401-kube-api-access-qznst\") pod \"kube-auth-proxy-5ff9c87565-srhn6\" (UID: \"5418a32d-6c58-4113-8ce4-52753a3d4401\") " pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" Jul 21 06:23:23.555322 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.555146 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5418a32d-6c58-4113-8ce4-52753a3d4401-tls-certs\") pod \"kube-auth-proxy-5ff9c87565-srhn6\" (UID: \"5418a32d-6c58-4113-8ce4-52753a3d4401\") " pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" Jul 21 06:23:23.656097 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.656056 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gcnl9"] Jul 21 06:23:23.656445 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.656420 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5418a32d-6c58-4113-8ce4-52753a3d4401-tls-certs\") pod \"kube-auth-proxy-5ff9c87565-srhn6\" (UID: \"5418a32d-6c58-4113-8ce4-52753a3d4401\") " pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" Jul 21 06:23:23.656523 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.656468 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5418a32d-6c58-4113-8ce4-52753a3d4401-tmp\") pod \"kube-auth-proxy-5ff9c87565-srhn6\" (UID: \"5418a32d-6c58-4113-8ce4-52753a3d4401\") " pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" Jul 21 06:23:23.656523 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.656497 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qznst\" (UniqueName: \"kubernetes.io/projected/5418a32d-6c58-4113-8ce4-52753a3d4401-kube-api-access-qznst\") pod \"kube-auth-proxy-5ff9c87565-srhn6\" (UID: \"5418a32d-6c58-4113-8ce4-52753a3d4401\") " pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" Jul 21 06:23:23.658881 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.658857 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5418a32d-6c58-4113-8ce4-52753a3d4401-tmp\") pod \"kube-auth-proxy-5ff9c87565-srhn6\" (UID: \"5418a32d-6c58-4113-8ce4-52753a3d4401\") " pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" Jul 21 06:23:23.659091 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.659073 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5418a32d-6c58-4113-8ce4-52753a3d4401-tls-certs\") pod \"kube-auth-proxy-5ff9c87565-srhn6\" (UID: \"5418a32d-6c58-4113-8ce4-52753a3d4401\") " pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" Jul 21 06:23:23.659566 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.659551 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" Jul 21 06:23:23.662384 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.662367 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 21 06:23:23.662475 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.662386 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-hzg4p\"" Jul 21 06:23:23.670742 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.670719 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gcnl9"] Jul 21 06:23:23.672371 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.672350 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qznst\" (UniqueName: \"kubernetes.io/projected/5418a32d-6c58-4113-8ce4-52753a3d4401-kube-api-access-qznst\") pod \"kube-auth-proxy-5ff9c87565-srhn6\" (UID: \"5418a32d-6c58-4113-8ce4-52753a3d4401\") " pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" Jul 21 06:23:23.757915 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.757829 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c821d0c2-52a7-4253-9dfe-1b6e7dc709ae-cert\") pod \"odh-model-controller-7998c67dc4-gcnl9\" (UID: \"c821d0c2-52a7-4253-9dfe-1b6e7dc709ae\") " pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" Jul 21 06:23:23.758067 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.757947 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28s6x\" (UniqueName: \"kubernetes.io/projected/c821d0c2-52a7-4253-9dfe-1b6e7dc709ae-kube-api-access-28s6x\") pod \"odh-model-controller-7998c67dc4-gcnl9\" (UID: \"c821d0c2-52a7-4253-9dfe-1b6e7dc709ae\") " pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" Jul 21 06:23:23.768802 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.768772 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" Jul 21 06:23:23.858763 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.858729 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28s6x\" (UniqueName: \"kubernetes.io/projected/c821d0c2-52a7-4253-9dfe-1b6e7dc709ae-kube-api-access-28s6x\") pod \"odh-model-controller-7998c67dc4-gcnl9\" (UID: \"c821d0c2-52a7-4253-9dfe-1b6e7dc709ae\") " pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" Jul 21 06:23:23.858910 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.858784 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c821d0c2-52a7-4253-9dfe-1b6e7dc709ae-cert\") pod \"odh-model-controller-7998c67dc4-gcnl9\" (UID: \"c821d0c2-52a7-4253-9dfe-1b6e7dc709ae\") " pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" Jul 21 06:23:23.858971 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:23:23.858922 2563 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 21 06:23:23.859026 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:23:23.858975 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c821d0c2-52a7-4253-9dfe-1b6e7dc709ae-cert podName:c821d0c2-52a7-4253-9dfe-1b6e7dc709ae nodeName:}" failed. No retries permitted until 2026-07-21 06:23:24.358959963 +0000 UTC m=+334.304598599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c821d0c2-52a7-4253-9dfe-1b6e7dc709ae-cert") pod "odh-model-controller-7998c67dc4-gcnl9" (UID: "c821d0c2-52a7-4253-9dfe-1b6e7dc709ae") : secret "odh-model-controller-webhook-cert" not found Jul 21 06:23:23.873051 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.872991 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28s6x\" (UniqueName: \"kubernetes.io/projected/c821d0c2-52a7-4253-9dfe-1b6e7dc709ae-kube-api-access-28s6x\") pod \"odh-model-controller-7998c67dc4-gcnl9\" (UID: \"c821d0c2-52a7-4253-9dfe-1b6e7dc709ae\") " pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" Jul 21 06:23:23.907339 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:23.907311 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6"] Jul 21 06:23:23.909107 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:23:23.909083 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5418a32d_6c58_4113_8ce4_52753a3d4401.slice/crio-2293f1452cc70903613b72143051638ef3a4cdca6c17db0b37ced499e257a718 WatchSource:0}: Error finding container 2293f1452cc70903613b72143051638ef3a4cdca6c17db0b37ced499e257a718: Status 404 returned error can't find the container with id 2293f1452cc70903613b72143051638ef3a4cdca6c17db0b37ced499e257a718 Jul 21 06:23:24.178518 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:24.178485 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" event={"ID":"5418a32d-6c58-4113-8ce4-52753a3d4401","Type":"ContainerStarted","Data":"2293f1452cc70903613b72143051638ef3a4cdca6c17db0b37ced499e257a718"} Jul 21 06:23:24.363922 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:24.363888 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c821d0c2-52a7-4253-9dfe-1b6e7dc709ae-cert\") pod \"odh-model-controller-7998c67dc4-gcnl9\" (UID: \"c821d0c2-52a7-4253-9dfe-1b6e7dc709ae\") " pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" Jul 21 06:23:24.366523 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:24.366498 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c821d0c2-52a7-4253-9dfe-1b6e7dc709ae-cert\") pod \"odh-model-controller-7998c67dc4-gcnl9\" (UID: \"c821d0c2-52a7-4253-9dfe-1b6e7dc709ae\") " pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" Jul 21 06:23:24.587574 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:24.587542 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" Jul 21 06:23:24.770687 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:24.770661 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-gcnl9"] Jul 21 06:23:24.773474 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:23:24.773436 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc821d0c2_52a7_4253_9dfe_1b6e7dc709ae.slice/crio-ddbbad48c18dcac81055b942f1a87f0a77b2f58fd3856e4f74878883780f0740 WatchSource:0}: Error finding container ddbbad48c18dcac81055b942f1a87f0a77b2f58fd3856e4f74878883780f0740: Status 404 returned error can't find the container with id ddbbad48c18dcac81055b942f1a87f0a77b2f58fd3856e4f74878883780f0740 Jul 21 06:23:25.183984 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:25.183949 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" event={"ID":"c821d0c2-52a7-4253-9dfe-1b6e7dc709ae","Type":"ContainerStarted","Data":"ddbbad48c18dcac81055b942f1a87f0a77b2f58fd3856e4f74878883780f0740"} Jul 21 06:23:28.201733 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:28.201692 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" event={"ID":"5418a32d-6c58-4113-8ce4-52753a3d4401","Type":"ContainerStarted","Data":"8470057e594a399570b7627fda79f577a882d4fdeed8c3e212d1cc7a4bef3b5b"} Jul 21 06:23:28.203508 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:28.203482 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" event={"ID":"c821d0c2-52a7-4253-9dfe-1b6e7dc709ae","Type":"ContainerStarted","Data":"2651c8afb81faed2e1839afaf927e9977777bd42c4469623446d7d2bd14d0903"} Jul 21 06:23:28.203627 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:28.203613 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" Jul 21 06:23:28.218258 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:28.218187 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5ff9c87565-srhn6" podStartSLOduration=1.352110668 podStartE2EDuration="5.218175533s" totalImagesPullingTime="3.866064865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:23:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:23:28.21742961 +0000 UTC m=+338.163068265" watchObservedRunningTime="2026-07-21 06:23:28.218175533 +0000 UTC m=+338.163814190" Jul 21 06:23:28.233080 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:28.233029 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" podStartSLOduration=2.180584988 podStartE2EDuration="5.233010477s" totalImagesPullingTime="3.052425489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:23:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:23:28.23241463 +0000 UTC m=+338.178053313" watchObservedRunningTime="2026-07-21 06:23:28.233010477 +0000 UTC m=+338.178649135" Jul 21 06:23:29.842180 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.842150 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc"] Jul 21 06:23:29.845594 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.845576 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:29.849166 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.849135 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-pzt68\"" Jul 21 06:23:29.849291 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.849229 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 21 06:23:29.863430 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.863408 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc"] Jul 21 06:23:29.915383 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.915345 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:29.915578 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.915388 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:29.915578 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.915424 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:29.915578 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.915448 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:29.915578 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.915473 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9l6p\" (UniqueName: \"kubernetes.io/projected/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-kube-api-access-p9l6p\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:29.915578 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.915565 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:29.915804 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.915591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:29.915804 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.915634 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:29.915804 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:29.915662 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.017094 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.017010 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.017094 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.017049 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.017094 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.017075 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.017446 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.017096 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.017446 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.017127 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9l6p\" (UniqueName: \"kubernetes.io/projected/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-kube-api-access-p9l6p\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.017446 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.017211 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.017446 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.017266 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.017446 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.017313 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.017446 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.017342 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.017763 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.017564 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.017763 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.017647 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.017763 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.017744 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.017979 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.017950 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.018074 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.018053 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.020251 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.020205 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.020457 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.020436 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.028995 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.028975 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.031749 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.031726 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9l6p\" (UniqueName: \"kubernetes.io/projected/cbe60fb0-7883-43a9-be69-39cfaa4b3c5c-kube-api-access-p9l6p\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rxdmc\" (UID: \"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.155368 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.155335 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:30.292622 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:30.292595 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc"] Jul 21 06:23:30.293779 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:23:30.293756 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbe60fb0_7883_43a9_be69_39cfaa4b3c5c.slice/crio-1d55888029fa0efc63299163c897b3def4d0f2ab192340b4bb0c21a7804377ce WatchSource:0}: Error finding container 1d55888029fa0efc63299163c897b3def4d0f2ab192340b4bb0c21a7804377ce: Status 404 returned error can't find the container with id 1d55888029fa0efc63299163c897b3def4d0f2ab192340b4bb0c21a7804377ce Jul 21 06:23:31.221473 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.221342 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" event={"ID":"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c","Type":"ContainerStarted","Data":"1d55888029fa0efc63299163c897b3def4d0f2ab192340b4bb0c21a7804377ce"} Jul 21 06:23:31.650347 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.650305 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs"] Jul 21 06:23:31.653908 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.653884 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.656926 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.656900 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 21 06:23:31.657056 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.656937 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 21 06:23:31.657056 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.656901 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 21 06:23:31.657205 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.657187 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-c8khv\"" Jul 21 06:23:31.664759 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.664733 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs"] Jul 21 06:23:31.735588 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.735558 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f535186-b6ab-49c5-90e3-a629cfbed20f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.735853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.735590 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4f535186-b6ab-49c5-90e3-a629cfbed20f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.735853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.735623 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4f535186-b6ab-49c5-90e3-a629cfbed20f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.735853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.735691 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f535186-b6ab-49c5-90e3-a629cfbed20f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.735853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.735746 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4f535186-b6ab-49c5-90e3-a629cfbed20f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.735853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.735838 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f535186-b6ab-49c5-90e3-a629cfbed20f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.735853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.735868 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whfkw\" (UniqueName: \"kubernetes.io/projected/4f535186-b6ab-49c5-90e3-a629cfbed20f-kube-api-access-whfkw\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.837205 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.837166 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f535186-b6ab-49c5-90e3-a629cfbed20f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.837398 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.837226 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4f535186-b6ab-49c5-90e3-a629cfbed20f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.837398 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.837255 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4f535186-b6ab-49c5-90e3-a629cfbed20f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.837398 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.837286 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f535186-b6ab-49c5-90e3-a629cfbed20f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.837398 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.837307 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4f535186-b6ab-49c5-90e3-a629cfbed20f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.837398 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.837359 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f535186-b6ab-49c5-90e3-a629cfbed20f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.837398 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.837380 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whfkw\" (UniqueName: \"kubernetes.io/projected/4f535186-b6ab-49c5-90e3-a629cfbed20f-kube-api-access-whfkw\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.838596 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.838568 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4f535186-b6ab-49c5-90e3-a629cfbed20f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.840078 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.840051 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f535186-b6ab-49c5-90e3-a629cfbed20f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.840861 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.840838 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4f535186-b6ab-49c5-90e3-a629cfbed20f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.840941 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.840837 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4f535186-b6ab-49c5-90e3-a629cfbed20f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.840941 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.840893 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4f535186-b6ab-49c5-90e3-a629cfbed20f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.846944 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.846917 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f535186-b6ab-49c5-90e3-a629cfbed20f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.847492 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.847471 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whfkw\" (UniqueName: \"kubernetes.io/projected/4f535186-b6ab-49c5-90e3-a629cfbed20f-kube-api-access-whfkw\") pod \"istiod-openshift-gateway-94bb8fbfd-2dvfs\" (UID: \"4f535186-b6ab-49c5-90e3-a629cfbed20f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:31.968733 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:31.968664 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:32.988798 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:32.988757 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:23:32.989092 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:32.988849 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:23:32.989092 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:32.988896 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:23:33.069885 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.069861 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs"] Jul 21 06:23:33.070562 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:23:33.070539 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f535186_b6ab_49c5_90e3_a629cfbed20f.slice/crio-d7a1ade27829be32730fc502fe60e92d76c762ab373397360a546c6e4008b236 WatchSource:0}: Error finding container d7a1ade27829be32730fc502fe60e92d76c762ab373397360a546c6e4008b236: Status 404 returned error can't find the container with id d7a1ade27829be32730fc502fe60e92d76c762ab373397360a546c6e4008b236 Jul 21 06:23:33.234419 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.234334 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" event={"ID":"cbe60fb0-7883-43a9-be69-39cfaa4b3c5c","Type":"ContainerStarted","Data":"c32db11abbfb5a75e5552b8a33bf42b16cc87591f7778b855b5935c9cd3eae75"} Jul 21 06:23:33.235648 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.235627 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" event={"ID":"4f535186-b6ab-49c5-90e3-a629cfbed20f","Type":"ContainerStarted","Data":"d7a1ade27829be32730fc502fe60e92d76c762ab373397360a546c6e4008b236"} Jul 21 06:23:33.272025 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.271975 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" podStartSLOduration=1.579343743 podStartE2EDuration="4.271958742s" totalImagesPullingTime="2.692614999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:23:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:23:33.270731479 +0000 UTC m=+343.216370160" watchObservedRunningTime="2026-07-21 06:23:33.271958742 +0000 UTC m=+343.217597398" Jul 21 06:23:33.593884 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.593819 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-99gxr"] Jul 21 06:23:33.597890 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.597866 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-99gxr" Jul 21 06:23:33.601272 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.601238 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-wtcmc\"" Jul 21 06:23:33.601413 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.601332 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 21 06:23:33.607511 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.607023 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-99gxr"] Jul 21 06:23:33.612020 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.612000 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv"] Jul 21 06:23:33.615514 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.615491 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" Jul 21 06:23:33.618495 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.618455 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 21 06:23:33.618655 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.618494 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-xblpz\"" Jul 21 06:23:33.627846 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.627807 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv"] Jul 21 06:23:33.655428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.655398 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce0b4c40-cbda-4526-8404-69090e199629-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w6btv\" (UID: \"ce0b4c40-cbda-4526-8404-69090e199629\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" Jul 21 06:23:33.655428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.655431 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d95d08d0-1a92-457d-98d8-6e347fe04d32-cert\") pod \"kserve-controller-manager-d6f4b65b8-99gxr\" (UID: \"d95d08d0-1a92-457d-98d8-6e347fe04d32\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-99gxr" Jul 21 06:23:33.655627 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.655477 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwcnb\" (UniqueName: \"kubernetes.io/projected/ce0b4c40-cbda-4526-8404-69090e199629-kube-api-access-hwcnb\") pod \"llmisvc-controller-manager-777c6bd7d5-w6btv\" (UID: \"ce0b4c40-cbda-4526-8404-69090e199629\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" Jul 21 06:23:33.655627 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.655503 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65w6s\" (UniqueName: \"kubernetes.io/projected/d95d08d0-1a92-457d-98d8-6e347fe04d32-kube-api-access-65w6s\") pod \"kserve-controller-manager-d6f4b65b8-99gxr\" (UID: \"d95d08d0-1a92-457d-98d8-6e347fe04d32\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-99gxr" Jul 21 06:23:33.757009 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.756970 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d95d08d0-1a92-457d-98d8-6e347fe04d32-cert\") pod \"kserve-controller-manager-d6f4b65b8-99gxr\" (UID: \"d95d08d0-1a92-457d-98d8-6e347fe04d32\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-99gxr" Jul 21 06:23:33.757177 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.757032 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwcnb\" (UniqueName: \"kubernetes.io/projected/ce0b4c40-cbda-4526-8404-69090e199629-kube-api-access-hwcnb\") pod \"llmisvc-controller-manager-777c6bd7d5-w6btv\" (UID: \"ce0b4c40-cbda-4526-8404-69090e199629\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" Jul 21 06:23:33.757287 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.757266 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65w6s\" (UniqueName: \"kubernetes.io/projected/d95d08d0-1a92-457d-98d8-6e347fe04d32-kube-api-access-65w6s\") pod \"kserve-controller-manager-d6f4b65b8-99gxr\" (UID: \"d95d08d0-1a92-457d-98d8-6e347fe04d32\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-99gxr" Jul 21 06:23:33.757422 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.757403 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce0b4c40-cbda-4526-8404-69090e199629-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w6btv\" (UID: \"ce0b4c40-cbda-4526-8404-69090e199629\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" Jul 21 06:23:33.757531 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:23:33.757516 2563 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 21 06:23:33.757593 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:23:33.757577 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0b4c40-cbda-4526-8404-69090e199629-cert podName:ce0b4c40-cbda-4526-8404-69090e199629 nodeName:}" failed. No retries permitted until 2026-07-21 06:23:34.257557585 +0000 UTC m=+344.203196230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ce0b4c40-cbda-4526-8404-69090e199629-cert") pod "llmisvc-controller-manager-777c6bd7d5-w6btv" (UID: "ce0b4c40-cbda-4526-8404-69090e199629") : secret "llmisvc-webhook-server-cert" not found Jul 21 06:23:33.760121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.760089 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d95d08d0-1a92-457d-98d8-6e347fe04d32-cert\") pod \"kserve-controller-manager-d6f4b65b8-99gxr\" (UID: \"d95d08d0-1a92-457d-98d8-6e347fe04d32\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-99gxr" Jul 21 06:23:33.773704 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.773672 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwcnb\" (UniqueName: \"kubernetes.io/projected/ce0b4c40-cbda-4526-8404-69090e199629-kube-api-access-hwcnb\") pod \"llmisvc-controller-manager-777c6bd7d5-w6btv\" (UID: \"ce0b4c40-cbda-4526-8404-69090e199629\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" Jul 21 06:23:33.773908 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.773886 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65w6s\" (UniqueName: \"kubernetes.io/projected/d95d08d0-1a92-457d-98d8-6e347fe04d32-kube-api-access-65w6s\") pod \"kserve-controller-manager-d6f4b65b8-99gxr\" (UID: \"d95d08d0-1a92-457d-98d8-6e347fe04d32\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-99gxr" Jul 21 06:23:33.912641 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:33.912546 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-99gxr" Jul 21 06:23:34.084813 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:34.084713 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-99gxr"] Jul 21 06:23:34.087376 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:23:34.087348 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95d08d0_1a92_457d_98d8_6e347fe04d32.slice/crio-9c3db161f90eea68c294417c2c2f25c71e428ccab561dec9553959d27eff2c4a WatchSource:0}: Error finding container 9c3db161f90eea68c294417c2c2f25c71e428ccab561dec9553959d27eff2c4a: Status 404 returned error can't find the container with id 9c3db161f90eea68c294417c2c2f25c71e428ccab561dec9553959d27eff2c4a Jul 21 06:23:34.156343 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:34.156162 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:34.161836 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:34.161802 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:34.246170 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:34.246027 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-99gxr" event={"ID":"d95d08d0-1a92-457d-98d8-6e347fe04d32","Type":"ContainerStarted","Data":"9c3db161f90eea68c294417c2c2f25c71e428ccab561dec9553959d27eff2c4a"} Jul 21 06:23:34.246375 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:34.246204 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:34.247438 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:34.247414 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rxdmc" Jul 21 06:23:34.262722 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:34.262692 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce0b4c40-cbda-4526-8404-69090e199629-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w6btv\" (UID: \"ce0b4c40-cbda-4526-8404-69090e199629\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" Jul 21 06:23:34.265544 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:34.265519 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce0b4c40-cbda-4526-8404-69090e199629-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w6btv\" (UID: \"ce0b4c40-cbda-4526-8404-69090e199629\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" Jul 21 06:23:34.530650 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:34.530575 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" Jul 21 06:23:35.377485 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:35.377449 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:23:35.377786 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:35.377513 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:23:35.445338 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:35.445315 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv"] Jul 21 06:23:35.446862 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:23:35.446839 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podce0b4c40_cbda_4526_8404_69090e199629.slice/crio-28260a030ad3b9036ec255931a47b7df4f6ce07375df67c3f1c1e7d4de230707 WatchSource:0}: Error finding container 28260a030ad3b9036ec255931a47b7df4f6ce07375df67c3f1c1e7d4de230707: Status 404 returned error can't find the container with id 28260a030ad3b9036ec255931a47b7df4f6ce07375df67c3f1c1e7d4de230707 Jul 21 06:23:36.260866 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:36.260441 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" event={"ID":"ce0b4c40-cbda-4526-8404-69090e199629","Type":"ContainerStarted","Data":"28260a030ad3b9036ec255931a47b7df4f6ce07375df67c3f1c1e7d4de230707"} Jul 21 06:23:36.262299 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:36.262266 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" event={"ID":"4f535186-b6ab-49c5-90e3-a629cfbed20f","Type":"ContainerStarted","Data":"cef4c0d0a414c98c9a310ec1ababc22f2850ea42b4411081ee2747e0bb299834"} Jul 21 06:23:36.262618 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:36.262575 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:36.264486 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:36.264467 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" Jul 21 06:23:36.287571 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:36.287513 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2dvfs" podStartSLOduration=2.9830186100000002 podStartE2EDuration="5.287493541s" totalImagesPullingTime="2.304474931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:23:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:23:36.285346794 +0000 UTC m=+346.230985451" watchObservedRunningTime="2026-07-21 06:23:36.287493541 +0000 UTC m=+346.233132198" Jul 21 06:23:38.274673 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:38.274641 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-99gxr" event={"ID":"d95d08d0-1a92-457d-98d8-6e347fe04d32","Type":"ContainerStarted","Data":"e20291eda9e16f1e8bc46aa031a126cbf86980b8587a839f17763821eb6c8761"} Jul 21 06:23:38.275112 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:38.274854 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-99gxr" Jul 21 06:23:38.304637 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:38.304590 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-99gxr" podStartSLOduration=1.9761342339999999 podStartE2EDuration="5.304573896s" totalImagesPullingTime="3.328439662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:23:38.303484025 +0000 UTC m=+348.249122704" watchObservedRunningTime="2026-07-21 06:23:38.304573896 +0000 UTC m=+348.250212554" Jul 21 06:23:39.209830 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:39.209799 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-gcnl9" Jul 21 06:23:39.282607 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:39.282569 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" event={"ID":"ce0b4c40-cbda-4526-8404-69090e199629","Type":"ContainerStarted","Data":"559740928a93b1793ad309c3a646d782244563eea64f13e1d665ad576f864f7f"} Jul 21 06:23:39.283036 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:39.282654 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" Jul 21 06:23:39.299573 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:23:39.299525 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" podStartSLOduration=3.412131175 podStartE2EDuration="6.299510213s" totalImagesPullingTime="2.887379038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:23:39.297956977 +0000 UTC m=+349.243595634" watchObservedRunningTime="2026-07-21 06:23:39.299510213 +0000 UTC m=+349.245148870" Jul 21 06:24:09.288208 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:09.288128 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-99gxr" Jul 21 06:24:10.289359 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:10.289325 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w6btv" Jul 21 06:24:36.349916 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.349876 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx"] Jul 21 06:24:36.353226 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.353200 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" Jul 21 06:24:36.356101 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.356082 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 06:24:36.356248 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.356085 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jf2qz\"" Jul 21 06:24:36.357672 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.357650 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 06:24:36.361286 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.361266 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx"] Jul 21 06:24:36.401628 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.401589 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f305a214-c729-4ed3-b051-3611cc3b061d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx\" (UID: \"f305a214-c729-4ed3-b051-3611cc3b061d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" Jul 21 06:24:36.401628 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.401635 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f305a214-c729-4ed3-b051-3611cc3b061d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx\" (UID: \"f305a214-c729-4ed3-b051-3611cc3b061d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" Jul 21 06:24:36.401864 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.401765 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj94l\" (UniqueName: \"kubernetes.io/projected/f305a214-c729-4ed3-b051-3611cc3b061d-kube-api-access-mj94l\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx\" (UID: \"f305a214-c729-4ed3-b051-3611cc3b061d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" Jul 21 06:24:36.502795 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.502755 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj94l\" (UniqueName: \"kubernetes.io/projected/f305a214-c729-4ed3-b051-3611cc3b061d-kube-api-access-mj94l\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx\" (UID: \"f305a214-c729-4ed3-b051-3611cc3b061d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" Jul 21 06:24:36.502958 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.502850 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f305a214-c729-4ed3-b051-3611cc3b061d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx\" (UID: \"f305a214-c729-4ed3-b051-3611cc3b061d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" Jul 21 06:24:36.502958 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.502881 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f305a214-c729-4ed3-b051-3611cc3b061d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx\" (UID: \"f305a214-c729-4ed3-b051-3611cc3b061d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" Jul 21 06:24:36.503291 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.503271 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f305a214-c729-4ed3-b051-3611cc3b061d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx\" (UID: \"f305a214-c729-4ed3-b051-3611cc3b061d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" Jul 21 06:24:36.503336 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.503307 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f305a214-c729-4ed3-b051-3611cc3b061d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx\" (UID: \"f305a214-c729-4ed3-b051-3611cc3b061d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" Jul 21 06:24:36.511090 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.511060 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj94l\" (UniqueName: \"kubernetes.io/projected/f305a214-c729-4ed3-b051-3611cc3b061d-kube-api-access-mj94l\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx\" (UID: \"f305a214-c729-4ed3-b051-3611cc3b061d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" Jul 21 06:24:36.663148 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.663056 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" Jul 21 06:24:36.798539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.798513 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx"] Jul 21 06:24:36.799634 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:24:36.799606 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf305a214_c729_4ed3_b051_3611cc3b061d.slice/crio-6eb60b5bd8a70886e4ad234f6ebbf8e6bb2b0e50cfa604680c367e385dfec229 WatchSource:0}: Error finding container 6eb60b5bd8a70886e4ad234f6ebbf8e6bb2b0e50cfa604680c367e385dfec229: Status 404 returned error can't find the container with id 6eb60b5bd8a70886e4ad234f6ebbf8e6bb2b0e50cfa604680c367e385dfec229 Jul 21 06:24:36.958694 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.958616 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn"] Jul 21 06:24:36.961959 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.961943 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" Jul 21 06:24:36.968897 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:36.968870 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn"] Jul 21 06:24:37.007991 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.007951 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t47n\" (UniqueName: \"kubernetes.io/projected/a1e61a69-b7fe-4663-ba0f-451a52b630af-kube-api-access-6t47n\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn\" (UID: \"a1e61a69-b7fe-4663-ba0f-451a52b630af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" Jul 21 06:24:37.008157 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.008016 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1e61a69-b7fe-4663-ba0f-451a52b630af-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn\" (UID: \"a1e61a69-b7fe-4663-ba0f-451a52b630af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" Jul 21 06:24:37.008157 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.008084 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1e61a69-b7fe-4663-ba0f-451a52b630af-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn\" (UID: \"a1e61a69-b7fe-4663-ba0f-451a52b630af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" Jul 21 06:24:37.109417 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.109372 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t47n\" (UniqueName: \"kubernetes.io/projected/a1e61a69-b7fe-4663-ba0f-451a52b630af-kube-api-access-6t47n\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn\" (UID: \"a1e61a69-b7fe-4663-ba0f-451a52b630af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" Jul 21 06:24:37.109616 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.109438 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1e61a69-b7fe-4663-ba0f-451a52b630af-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn\" (UID: \"a1e61a69-b7fe-4663-ba0f-451a52b630af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" Jul 21 06:24:37.109616 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.109501 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1e61a69-b7fe-4663-ba0f-451a52b630af-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn\" (UID: \"a1e61a69-b7fe-4663-ba0f-451a52b630af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" Jul 21 06:24:37.109857 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.109838 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1e61a69-b7fe-4663-ba0f-451a52b630af-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn\" (UID: \"a1e61a69-b7fe-4663-ba0f-451a52b630af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" Jul 21 06:24:37.109901 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.109858 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1e61a69-b7fe-4663-ba0f-451a52b630af-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn\" (UID: \"a1e61a69-b7fe-4663-ba0f-451a52b630af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" Jul 21 06:24:37.117991 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.117970 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t47n\" (UniqueName: \"kubernetes.io/projected/a1e61a69-b7fe-4663-ba0f-451a52b630af-kube-api-access-6t47n\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn\" (UID: \"a1e61a69-b7fe-4663-ba0f-451a52b630af\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" Jul 21 06:24:37.276043 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.275953 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" Jul 21 06:24:37.408603 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.408577 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn"] Jul 21 06:24:37.410521 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:24:37.410500 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1e61a69_b7fe_4663_ba0f_451a52b630af.slice/crio-104208ab9ddb8a925d14e2cbfce476f73cea151cd8039025362ef02b253b1919 WatchSource:0}: Error finding container 104208ab9ddb8a925d14e2cbfce476f73cea151cd8039025362ef02b253b1919: Status 404 returned error can't find the container with id 104208ab9ddb8a925d14e2cbfce476f73cea151cd8039025362ef02b253b1919 Jul 21 06:24:37.558129 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.557856 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5"] Jul 21 06:24:37.561812 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.561786 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" Jul 21 06:24:37.563851 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.563828 2563 generic.go:356] "Generic (PLEG): container finished" podID="a1e61a69-b7fe-4663-ba0f-451a52b630af" containerID="85473f12479df54ff848444b6d0f952168a630cd0f36cd61d4cd2da8015460a5" exitCode=0 Jul 21 06:24:37.563967 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.563912 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" event={"ID":"a1e61a69-b7fe-4663-ba0f-451a52b630af","Type":"ContainerDied","Data":"85473f12479df54ff848444b6d0f952168a630cd0f36cd61d4cd2da8015460a5"} Jul 21 06:24:37.563967 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.563961 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" event={"ID":"a1e61a69-b7fe-4663-ba0f-451a52b630af","Type":"ContainerStarted","Data":"104208ab9ddb8a925d14e2cbfce476f73cea151cd8039025362ef02b253b1919"} Jul 21 06:24:37.566238 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.566186 2563 generic.go:356] "Generic (PLEG): container finished" podID="f305a214-c729-4ed3-b051-3611cc3b061d" containerID="4c360789ce774f41a7cf04c4c84c9e98dd93bc2e6022223bd9823efab50fba68" exitCode=0 Jul 21 06:24:37.566358 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.566210 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" event={"ID":"f305a214-c729-4ed3-b051-3611cc3b061d","Type":"ContainerDied","Data":"4c360789ce774f41a7cf04c4c84c9e98dd93bc2e6022223bd9823efab50fba68"} Jul 21 06:24:37.566358 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.566287 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" event={"ID":"f305a214-c729-4ed3-b051-3611cc3b061d","Type":"ContainerStarted","Data":"6eb60b5bd8a70886e4ad234f6ebbf8e6bb2b0e50cfa604680c367e385dfec229"} Jul 21 06:24:37.568834 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.568813 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5"] Jul 21 06:24:37.614339 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.614306 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5\" (UID: \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" Jul 21 06:24:37.614530 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.614395 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5\" (UID: \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" Jul 21 06:24:37.614530 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.614481 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-279zk\" (UniqueName: \"kubernetes.io/projected/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-kube-api-access-279zk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5\" (UID: \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" Jul 21 06:24:37.715413 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.715374 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5\" (UID: \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" Jul 21 06:24:37.715605 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.715436 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5\" (UID: \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" Jul 21 06:24:37.715605 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.715473 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-279zk\" (UniqueName: \"kubernetes.io/projected/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-kube-api-access-279zk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5\" (UID: \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" Jul 21 06:24:37.715867 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.715836 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5\" (UID: \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" Jul 21 06:24:37.715939 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.715844 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5\" (UID: \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" Jul 21 06:24:37.723881 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.723858 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-279zk\" (UniqueName: \"kubernetes.io/projected/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-kube-api-access-279zk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5\" (UID: \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" Jul 21 06:24:37.873718 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:37.873684 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" Jul 21 06:24:38.007822 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.007799 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5"] Jul 21 06:24:38.008803 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:24:38.008776 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cad27d4_5fce_4a5e_b2a5_100d1ae5632d.slice/crio-a4024c2ca09fef77e9d27835b7a4191ebd9902babb61f1c14b6ce37735f97477 WatchSource:0}: Error finding container a4024c2ca09fef77e9d27835b7a4191ebd9902babb61f1c14b6ce37735f97477: Status 404 returned error can't find the container with id a4024c2ca09fef77e9d27835b7a4191ebd9902babb61f1c14b6ce37735f97477 Jul 21 06:24:38.149284 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.149193 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4"] Jul 21 06:24:38.152878 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.152861 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" Jul 21 06:24:38.162508 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.162486 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4"] Jul 21 06:24:38.220889 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.220857 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/011af3cc-80f2-4240-9519-fc4dc072d7e9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4\" (UID: \"011af3cc-80f2-4240-9519-fc4dc072d7e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" Jul 21 06:24:38.221038 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.220945 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ddl4\" (UniqueName: \"kubernetes.io/projected/011af3cc-80f2-4240-9519-fc4dc072d7e9-kube-api-access-8ddl4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4\" (UID: \"011af3cc-80f2-4240-9519-fc4dc072d7e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" Jul 21 06:24:38.221038 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.220981 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/011af3cc-80f2-4240-9519-fc4dc072d7e9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4\" (UID: \"011af3cc-80f2-4240-9519-fc4dc072d7e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" Jul 21 06:24:38.321532 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.321496 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/011af3cc-80f2-4240-9519-fc4dc072d7e9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4\" (UID: \"011af3cc-80f2-4240-9519-fc4dc072d7e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" Jul 21 06:24:38.321700 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.321588 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/011af3cc-80f2-4240-9519-fc4dc072d7e9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4\" (UID: \"011af3cc-80f2-4240-9519-fc4dc072d7e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" Jul 21 06:24:38.321772 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.321696 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ddl4\" (UniqueName: \"kubernetes.io/projected/011af3cc-80f2-4240-9519-fc4dc072d7e9-kube-api-access-8ddl4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4\" (UID: \"011af3cc-80f2-4240-9519-fc4dc072d7e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" Jul 21 06:24:38.321930 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.321905 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/011af3cc-80f2-4240-9519-fc4dc072d7e9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4\" (UID: \"011af3cc-80f2-4240-9519-fc4dc072d7e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" Jul 21 06:24:38.322233 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.322191 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/011af3cc-80f2-4240-9519-fc4dc072d7e9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4\" (UID: \"011af3cc-80f2-4240-9519-fc4dc072d7e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" Jul 21 06:24:38.333752 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.333729 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ddl4\" (UniqueName: \"kubernetes.io/projected/011af3cc-80f2-4240-9519-fc4dc072d7e9-kube-api-access-8ddl4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4\" (UID: \"011af3cc-80f2-4240-9519-fc4dc072d7e9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" Jul 21 06:24:38.477119 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.477086 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" Jul 21 06:24:38.574612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.574581 2563 generic.go:356] "Generic (PLEG): container finished" podID="a1e61a69-b7fe-4663-ba0f-451a52b630af" containerID="58505230aa624ac99fce528a34337c47dc46632a30818b03252c54af65f95b0a" exitCode=0 Jul 21 06:24:38.574757 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.574674 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" event={"ID":"a1e61a69-b7fe-4663-ba0f-451a52b630af","Type":"ContainerDied","Data":"58505230aa624ac99fce528a34337c47dc46632a30818b03252c54af65f95b0a"} Jul 21 06:24:38.576992 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.576966 2563 generic.go:356] "Generic (PLEG): container finished" podID="f305a214-c729-4ed3-b051-3611cc3b061d" containerID="7003757eaa2947568a1594f06f6fa48581dcf723695e56485bec90da05d016a5" exitCode=0 Jul 21 06:24:38.577115 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.577066 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" event={"ID":"f305a214-c729-4ed3-b051-3611cc3b061d","Type":"ContainerDied","Data":"7003757eaa2947568a1594f06f6fa48581dcf723695e56485bec90da05d016a5"} Jul 21 06:24:38.578911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.578807 2563 generic.go:356] "Generic (PLEG): container finished" podID="1cad27d4-5fce-4a5e-b2a5-100d1ae5632d" containerID="e8ad1d410d2cbff588f94c8ac54a7c1eb5b5ad43bcb80568062983e287038928" exitCode=0 Jul 21 06:24:38.578973 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.578936 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" event={"ID":"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d","Type":"ContainerDied","Data":"e8ad1d410d2cbff588f94c8ac54a7c1eb5b5ad43bcb80568062983e287038928"} Jul 21 06:24:38.579084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.578976 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" event={"ID":"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d","Type":"ContainerStarted","Data":"a4024c2ca09fef77e9d27835b7a4191ebd9902babb61f1c14b6ce37735f97477"} Jul 21 06:24:38.620564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:38.620535 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4"] Jul 21 06:24:38.628387 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:24:38.628365 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod011af3cc_80f2_4240_9519_fc4dc072d7e9.slice/crio-d4825ef31d157977bd2f637408e8397229484a9c0902c9d29467b164bfa20934 WatchSource:0}: Error finding container d4825ef31d157977bd2f637408e8397229484a9c0902c9d29467b164bfa20934: Status 404 returned error can't find the container with id d4825ef31d157977bd2f637408e8397229484a9c0902c9d29467b164bfa20934 Jul 21 06:24:39.587385 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:39.587357 2563 generic.go:356] "Generic (PLEG): container finished" podID="011af3cc-80f2-4240-9519-fc4dc072d7e9" containerID="f1793fd22b53546f48ccff5328a23b25097c6d09000f07cb151ea2145bdd1117" exitCode=0 Jul 21 06:24:39.587786 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:39.587440 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" event={"ID":"011af3cc-80f2-4240-9519-fc4dc072d7e9","Type":"ContainerDied","Data":"f1793fd22b53546f48ccff5328a23b25097c6d09000f07cb151ea2145bdd1117"} Jul 21 06:24:39.587786 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:39.587492 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" event={"ID":"011af3cc-80f2-4240-9519-fc4dc072d7e9","Type":"ContainerStarted","Data":"d4825ef31d157977bd2f637408e8397229484a9c0902c9d29467b164bfa20934"} Jul 21 06:24:39.590045 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:39.590022 2563 generic.go:356] "Generic (PLEG): container finished" podID="a1e61a69-b7fe-4663-ba0f-451a52b630af" containerID="f81ffe77d0b41e6d5e23174b885d76bfa1f429a20550e01fe0964871354a1b54" exitCode=0 Jul 21 06:24:39.590137 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:39.590116 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" event={"ID":"a1e61a69-b7fe-4663-ba0f-451a52b630af","Type":"ContainerDied","Data":"f81ffe77d0b41e6d5e23174b885d76bfa1f429a20550e01fe0964871354a1b54"} Jul 21 06:24:39.592459 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:39.592436 2563 generic.go:356] "Generic (PLEG): container finished" podID="f305a214-c729-4ed3-b051-3611cc3b061d" containerID="fa67ab24c23a4a7b67f4801725276feace6e0b09653f275ac44b3959e8975725" exitCode=0 Jul 21 06:24:39.592556 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:39.592486 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" event={"ID":"f305a214-c729-4ed3-b051-3611cc3b061d","Type":"ContainerDied","Data":"fa67ab24c23a4a7b67f4801725276feace6e0b09653f275ac44b3959e8975725"} Jul 21 06:24:40.612296 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.612254 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" event={"ID":"011af3cc-80f2-4240-9519-fc4dc072d7e9","Type":"ContainerStarted","Data":"98d6ab611c10854bbcbb2ace61437d9604462927ea8521c246128ceee127aa08"} Jul 21 06:24:40.614183 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.614159 2563 generic.go:356] "Generic (PLEG): container finished" podID="1cad27d4-5fce-4a5e-b2a5-100d1ae5632d" containerID="c039b738f09d02a04d6984daed3a07a8dae343170ccc56dd6c99bc430d6c6f8c" exitCode=0 Jul 21 06:24:40.614331 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.614247 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" event={"ID":"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d","Type":"ContainerDied","Data":"c039b738f09d02a04d6984daed3a07a8dae343170ccc56dd6c99bc430d6c6f8c"} Jul 21 06:24:40.774957 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.774934 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" Jul 21 06:24:40.799365 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.799336 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" Jul 21 06:24:40.846900 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.846874 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f305a214-c729-4ed3-b051-3611cc3b061d-bundle\") pod \"f305a214-c729-4ed3-b051-3611cc3b061d\" (UID: \"f305a214-c729-4ed3-b051-3611cc3b061d\") " Jul 21 06:24:40.847040 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.846909 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj94l\" (UniqueName: \"kubernetes.io/projected/f305a214-c729-4ed3-b051-3611cc3b061d-kube-api-access-mj94l\") pod \"f305a214-c729-4ed3-b051-3611cc3b061d\" (UID: \"f305a214-c729-4ed3-b051-3611cc3b061d\") " Jul 21 06:24:40.847040 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.846943 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t47n\" (UniqueName: \"kubernetes.io/projected/a1e61a69-b7fe-4663-ba0f-451a52b630af-kube-api-access-6t47n\") pod \"a1e61a69-b7fe-4663-ba0f-451a52b630af\" (UID: \"a1e61a69-b7fe-4663-ba0f-451a52b630af\") " Jul 21 06:24:40.847040 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.847010 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1e61a69-b7fe-4663-ba0f-451a52b630af-util\") pod \"a1e61a69-b7fe-4663-ba0f-451a52b630af\" (UID: \"a1e61a69-b7fe-4663-ba0f-451a52b630af\") " Jul 21 06:24:40.847040 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.847027 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1e61a69-b7fe-4663-ba0f-451a52b630af-bundle\") pod \"a1e61a69-b7fe-4663-ba0f-451a52b630af\" (UID: \"a1e61a69-b7fe-4663-ba0f-451a52b630af\") " Jul 21 06:24:40.847254 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.847052 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f305a214-c729-4ed3-b051-3611cc3b061d-util\") pod \"f305a214-c729-4ed3-b051-3611cc3b061d\" (UID: \"f305a214-c729-4ed3-b051-3611cc3b061d\") " Jul 21 06:24:40.847607 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.847581 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1e61a69-b7fe-4663-ba0f-451a52b630af-bundle" (OuterVolumeSpecName: "bundle") pod "a1e61a69-b7fe-4663-ba0f-451a52b630af" (UID: "a1e61a69-b7fe-4663-ba0f-451a52b630af"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:24:40.847909 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.847882 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f305a214-c729-4ed3-b051-3611cc3b061d-bundle" (OuterVolumeSpecName: "bundle") pod "f305a214-c729-4ed3-b051-3611cc3b061d" (UID: "f305a214-c729-4ed3-b051-3611cc3b061d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:24:40.849252 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.849211 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1e61a69-b7fe-4663-ba0f-451a52b630af-kube-api-access-6t47n" (OuterVolumeSpecName: "kube-api-access-6t47n") pod "a1e61a69-b7fe-4663-ba0f-451a52b630af" (UID: "a1e61a69-b7fe-4663-ba0f-451a52b630af"). InnerVolumeSpecName "kube-api-access-6t47n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:24:40.849424 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.849403 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f305a214-c729-4ed3-b051-3611cc3b061d-kube-api-access-mj94l" (OuterVolumeSpecName: "kube-api-access-mj94l") pod "f305a214-c729-4ed3-b051-3611cc3b061d" (UID: "f305a214-c729-4ed3-b051-3611cc3b061d"). InnerVolumeSpecName "kube-api-access-mj94l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:24:40.852983 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.852959 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f305a214-c729-4ed3-b051-3611cc3b061d-util" (OuterVolumeSpecName: "util") pod "f305a214-c729-4ed3-b051-3611cc3b061d" (UID: "f305a214-c729-4ed3-b051-3611cc3b061d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:24:40.853532 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.853513 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1e61a69-b7fe-4663-ba0f-451a52b630af-util" (OuterVolumeSpecName: "util") pod "a1e61a69-b7fe-4663-ba0f-451a52b630af" (UID: "a1e61a69-b7fe-4663-ba0f-451a52b630af"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:24:40.948584 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.948555 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a1e61a69-b7fe-4663-ba0f-451a52b630af-util\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:24:40.948584 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.948582 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a1e61a69-b7fe-4663-ba0f-451a52b630af-bundle\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:24:40.948584 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.948592 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f305a214-c729-4ed3-b051-3611cc3b061d-util\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:24:40.948799 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.948601 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f305a214-c729-4ed3-b051-3611cc3b061d-bundle\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:24:40.948799 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.948609 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj94l\" (UniqueName: \"kubernetes.io/projected/f305a214-c729-4ed3-b051-3611cc3b061d-kube-api-access-mj94l\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:24:40.948799 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:40.948620 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t47n\" (UniqueName: \"kubernetes.io/projected/a1e61a69-b7fe-4663-ba0f-451a52b630af-kube-api-access-6t47n\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:24:41.621704 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:41.621679 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" Jul 21 06:24:41.621704 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:41.621695 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn" event={"ID":"a1e61a69-b7fe-4663-ba0f-451a52b630af","Type":"ContainerDied","Data":"104208ab9ddb8a925d14e2cbfce476f73cea151cd8039025362ef02b253b1919"} Jul 21 06:24:41.622178 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:41.621738 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="104208ab9ddb8a925d14e2cbfce476f73cea151cd8039025362ef02b253b1919" Jul 21 06:24:41.623703 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:41.623681 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" event={"ID":"f305a214-c729-4ed3-b051-3611cc3b061d","Type":"ContainerDied","Data":"6eb60b5bd8a70886e4ad234f6ebbf8e6bb2b0e50cfa604680c367e385dfec229"} Jul 21 06:24:41.623800 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:41.623707 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx" Jul 21 06:24:41.623800 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:41.623721 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6eb60b5bd8a70886e4ad234f6ebbf8e6bb2b0e50cfa604680c367e385dfec229" Jul 21 06:24:41.625877 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:41.625852 2563 generic.go:356] "Generic (PLEG): container finished" podID="1cad27d4-5fce-4a5e-b2a5-100d1ae5632d" containerID="e1b3d5cb22285ff5db15b7cb489ad556caa99edba7c45060b0d484eeb0af0d89" exitCode=0 Jul 21 06:24:41.625993 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:41.625929 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" event={"ID":"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d","Type":"ContainerDied","Data":"e1b3d5cb22285ff5db15b7cb489ad556caa99edba7c45060b0d484eeb0af0d89"} Jul 21 06:24:41.628252 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:41.628230 2563 generic.go:356] "Generic (PLEG): container finished" podID="011af3cc-80f2-4240-9519-fc4dc072d7e9" containerID="98d6ab611c10854bbcbb2ace61437d9604462927ea8521c246128ceee127aa08" exitCode=0 Jul 21 06:24:41.628252 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:41.628247 2563 generic.go:356] "Generic (PLEG): container finished" podID="011af3cc-80f2-4240-9519-fc4dc072d7e9" containerID="d64d59eae10db56827203fabfb0862bf2114d0a2055b15667bda6eeb2d06d8c0" exitCode=0 Jul 21 06:24:41.628400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:41.628265 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" event={"ID":"011af3cc-80f2-4240-9519-fc4dc072d7e9","Type":"ContainerDied","Data":"98d6ab611c10854bbcbb2ace61437d9604462927ea8521c246128ceee127aa08"} Jul 21 06:24:41.628400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:41.628292 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" event={"ID":"011af3cc-80f2-4240-9519-fc4dc072d7e9","Type":"ContainerDied","Data":"d64d59eae10db56827203fabfb0862bf2114d0a2055b15667bda6eeb2d06d8c0"} Jul 21 06:24:42.770305 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.770277 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" Jul 21 06:24:42.799254 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.799228 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" Jul 21 06:24:42.865055 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.865027 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-bundle\") pod \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\" (UID: \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\") " Jul 21 06:24:42.865274 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.865087 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-util\") pod \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\" (UID: \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\") " Jul 21 06:24:42.865274 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.865114 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/011af3cc-80f2-4240-9519-fc4dc072d7e9-bundle\") pod \"011af3cc-80f2-4240-9519-fc4dc072d7e9\" (UID: \"011af3cc-80f2-4240-9519-fc4dc072d7e9\") " Jul 21 06:24:42.865274 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.865161 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ddl4\" (UniqueName: \"kubernetes.io/projected/011af3cc-80f2-4240-9519-fc4dc072d7e9-kube-api-access-8ddl4\") pod \"011af3cc-80f2-4240-9519-fc4dc072d7e9\" (UID: \"011af3cc-80f2-4240-9519-fc4dc072d7e9\") " Jul 21 06:24:42.865274 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.865245 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-279zk\" (UniqueName: \"kubernetes.io/projected/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-kube-api-access-279zk\") pod \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\" (UID: \"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d\") " Jul 21 06:24:42.865497 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.865390 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/011af3cc-80f2-4240-9519-fc4dc072d7e9-util\") pod \"011af3cc-80f2-4240-9519-fc4dc072d7e9\" (UID: \"011af3cc-80f2-4240-9519-fc4dc072d7e9\") " Jul 21 06:24:42.865582 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.865547 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-bundle" (OuterVolumeSpecName: "bundle") pod "1cad27d4-5fce-4a5e-b2a5-100d1ae5632d" (UID: "1cad27d4-5fce-4a5e-b2a5-100d1ae5632d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:24:42.865821 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.865795 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/011af3cc-80f2-4240-9519-fc4dc072d7e9-bundle" (OuterVolumeSpecName: "bundle") pod "011af3cc-80f2-4240-9519-fc4dc072d7e9" (UID: "011af3cc-80f2-4240-9519-fc4dc072d7e9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:24:42.865940 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.865896 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/011af3cc-80f2-4240-9519-fc4dc072d7e9-bundle\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:24:42.865940 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.865911 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-bundle\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:24:42.867490 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.867471 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-kube-api-access-279zk" (OuterVolumeSpecName: "kube-api-access-279zk") pod "1cad27d4-5fce-4a5e-b2a5-100d1ae5632d" (UID: "1cad27d4-5fce-4a5e-b2a5-100d1ae5632d"). InnerVolumeSpecName "kube-api-access-279zk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:24:42.867911 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.867897 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/011af3cc-80f2-4240-9519-fc4dc072d7e9-kube-api-access-8ddl4" (OuterVolumeSpecName: "kube-api-access-8ddl4") pod "011af3cc-80f2-4240-9519-fc4dc072d7e9" (UID: "011af3cc-80f2-4240-9519-fc4dc072d7e9"). InnerVolumeSpecName "kube-api-access-8ddl4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:24:42.871504 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.871480 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/011af3cc-80f2-4240-9519-fc4dc072d7e9-util" (OuterVolumeSpecName: "util") pod "011af3cc-80f2-4240-9519-fc4dc072d7e9" (UID: "011af3cc-80f2-4240-9519-fc4dc072d7e9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:24:42.873379 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.873358 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-util" (OuterVolumeSpecName: "util") pod "1cad27d4-5fce-4a5e-b2a5-100d1ae5632d" (UID: "1cad27d4-5fce-4a5e-b2a5-100d1ae5632d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:24:42.966869 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.966797 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-util\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:24:42.966869 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.966821 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ddl4\" (UniqueName: \"kubernetes.io/projected/011af3cc-80f2-4240-9519-fc4dc072d7e9-kube-api-access-8ddl4\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:24:42.966869 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.966832 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-279zk\" (UniqueName: \"kubernetes.io/projected/1cad27d4-5fce-4a5e-b2a5-100d1ae5632d-kube-api-access-279zk\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:24:42.966869 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:42.966841 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/011af3cc-80f2-4240-9519-fc4dc072d7e9-util\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:24:43.640045 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:43.640014 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" Jul 21 06:24:43.640211 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:43.640046 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5" event={"ID":"1cad27d4-5fce-4a5e-b2a5-100d1ae5632d","Type":"ContainerDied","Data":"a4024c2ca09fef77e9d27835b7a4191ebd9902babb61f1c14b6ce37735f97477"} Jul 21 06:24:43.640211 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:43.640099 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4024c2ca09fef77e9d27835b7a4191ebd9902babb61f1c14b6ce37735f97477" Jul 21 06:24:43.641978 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:43.641957 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" Jul 21 06:24:43.642092 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:43.641955 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4" event={"ID":"011af3cc-80f2-4240-9519-fc4dc072d7e9","Type":"ContainerDied","Data":"d4825ef31d157977bd2f637408e8397229484a9c0902c9d29467b164bfa20934"} Jul 21 06:24:43.642092 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:24:43.642067 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4825ef31d157977bd2f637408e8397229484a9c0902c9d29467b164bfa20934" Jul 21 06:25:03.641543 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:03.641503 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-855fc695dd-frncg"] Jul 21 06:25:04.633010 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.632981 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f"] Jul 21 06:25:04.633368 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633355 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="011af3cc-80f2-4240-9519-fc4dc072d7e9" containerName="extract" Jul 21 06:25:04.633428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633369 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="011af3cc-80f2-4240-9519-fc4dc072d7e9" containerName="extract" Jul 21 06:25:04.633428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633381 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f305a214-c729-4ed3-b051-3611cc3b061d" containerName="pull" Jul 21 06:25:04.633428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633386 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f305a214-c729-4ed3-b051-3611cc3b061d" containerName="pull" Jul 21 06:25:04.633428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633392 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="011af3cc-80f2-4240-9519-fc4dc072d7e9" containerName="util" Jul 21 06:25:04.633428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633397 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="011af3cc-80f2-4240-9519-fc4dc072d7e9" containerName="util" Jul 21 06:25:04.633428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633404 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cad27d4-5fce-4a5e-b2a5-100d1ae5632d" containerName="pull" Jul 21 06:25:04.633428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633409 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cad27d4-5fce-4a5e-b2a5-100d1ae5632d" containerName="pull" Jul 21 06:25:04.633428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633418 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f305a214-c729-4ed3-b051-3611cc3b061d" containerName="extract" Jul 21 06:25:04.633428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633422 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f305a214-c729-4ed3-b051-3611cc3b061d" containerName="extract" Jul 21 06:25:04.633428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633430 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1e61a69-b7fe-4663-ba0f-451a52b630af" containerName="util" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633434 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1e61a69-b7fe-4663-ba0f-451a52b630af" containerName="util" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633441 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f305a214-c729-4ed3-b051-3611cc3b061d" containerName="util" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633445 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f305a214-c729-4ed3-b051-3611cc3b061d" containerName="util" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633450 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cad27d4-5fce-4a5e-b2a5-100d1ae5632d" containerName="util" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633454 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cad27d4-5fce-4a5e-b2a5-100d1ae5632d" containerName="util" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633460 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1e61a69-b7fe-4663-ba0f-451a52b630af" containerName="extract" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633464 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1e61a69-b7fe-4663-ba0f-451a52b630af" containerName="extract" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633470 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="011af3cc-80f2-4240-9519-fc4dc072d7e9" containerName="pull" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633475 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="011af3cc-80f2-4240-9519-fc4dc072d7e9" containerName="pull" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633481 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1e61a69-b7fe-4663-ba0f-451a52b630af" containerName="pull" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633486 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1e61a69-b7fe-4663-ba0f-451a52b630af" containerName="pull" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633495 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cad27d4-5fce-4a5e-b2a5-100d1ae5632d" containerName="extract" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633500 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cad27d4-5fce-4a5e-b2a5-100d1ae5632d" containerName="extract" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633556 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f305a214-c729-4ed3-b051-3611cc3b061d" containerName="extract" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633562 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="011af3cc-80f2-4240-9519-fc4dc072d7e9" containerName="extract" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633572 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cad27d4-5fce-4a5e-b2a5-100d1ae5632d" containerName="extract" Jul 21 06:25:04.633709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.633577 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1e61a69-b7fe-4663-ba0f-451a52b630af" containerName="extract" Jul 21 06:25:04.637824 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.637808 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:04.641165 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.641135 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dsmhd\"" Jul 21 06:25:04.641301 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.641138 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 06:25:04.641301 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.641138 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 06:25:04.647102 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.647077 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f"] Jul 21 06:25:04.769104 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.769071 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcwq9\" (UniqueName: \"kubernetes.io/projected/165a355e-98c4-4485-a804-709cfb0205ba-kube-api-access-vcwq9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" (UID: \"165a355e-98c4-4485-a804-709cfb0205ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:04.769332 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.769200 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/165a355e-98c4-4485-a804-709cfb0205ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" (UID: \"165a355e-98c4-4485-a804-709cfb0205ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:04.870444 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.870405 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcwq9\" (UniqueName: \"kubernetes.io/projected/165a355e-98c4-4485-a804-709cfb0205ba-kube-api-access-vcwq9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" (UID: \"165a355e-98c4-4485-a804-709cfb0205ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:04.870626 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.870483 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/165a355e-98c4-4485-a804-709cfb0205ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" (UID: \"165a355e-98c4-4485-a804-709cfb0205ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:04.870892 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.870874 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/165a355e-98c4-4485-a804-709cfb0205ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" (UID: \"165a355e-98c4-4485-a804-709cfb0205ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:04.894545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.894468 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcwq9\" (UniqueName: \"kubernetes.io/projected/165a355e-98c4-4485-a804-709cfb0205ba-kube-api-access-vcwq9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" (UID: \"165a355e-98c4-4485-a804-709cfb0205ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:04.949370 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:04.949330 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:05.096114 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:05.096090 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f"] Jul 21 06:25:05.098790 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:25:05.098733 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod165a355e_98c4_4485_a804_709cfb0205ba.slice/crio-01b7a6e72065739bba8b1cb1033ca3fc3f8737df28e2cb142da792169ac7e523 WatchSource:0}: Error finding container 01b7a6e72065739bba8b1cb1033ca3fc3f8737df28e2cb142da792169ac7e523: Status 404 returned error can't find the container with id 01b7a6e72065739bba8b1cb1033ca3fc3f8737df28e2cb142da792169ac7e523 Jul 21 06:25:05.770953 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:05.770914 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" event={"ID":"165a355e-98c4-4485-a804-709cfb0205ba","Type":"ContainerStarted","Data":"01b7a6e72065739bba8b1cb1033ca3fc3f8737df28e2cb142da792169ac7e523"} Jul 21 06:25:11.811187 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:11.811092 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" event={"ID":"165a355e-98c4-4485-a804-709cfb0205ba","Type":"ContainerStarted","Data":"a82a5d2ba023fb8199dc0e335d0ae59734a23b8cf481b25b4e63bf761b4a0835"} Jul 21 06:25:11.811576 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:11.811201 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:11.834057 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:11.834011 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" podStartSLOduration=1.445081474 podStartE2EDuration="7.833997206s" totalImagesPullingTime="6.388915732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:25:11.8303156 +0000 UTC m=+441.775954256" watchObservedRunningTime="2026-07-21 06:25:11.833997206 +0000 UTC m=+441.779635864" Jul 21 06:25:22.819140 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:22.819109 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:24.227422 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.227386 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g"] Jul 21 06:25:24.232331 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.232314 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:24.245493 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.245466 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g"] Jul 21 06:25:24.339330 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.339295 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjd68\" (UniqueName: \"kubernetes.io/projected/bc930599-d1b5-4113-9d53-007089405527-kube-api-access-pjd68\") pod \"kuadrant-operator-controller-manager-55467d9f7-6ht6g\" (UID: \"bc930599-d1b5-4113-9d53-007089405527\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:24.339530 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.339353 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc930599-d1b5-4113-9d53-007089405527-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-6ht6g\" (UID: \"bc930599-d1b5-4113-9d53-007089405527\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:24.440627 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.440589 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc930599-d1b5-4113-9d53-007089405527-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-6ht6g\" (UID: \"bc930599-d1b5-4113-9d53-007089405527\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:24.440810 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.440679 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjd68\" (UniqueName: \"kubernetes.io/projected/bc930599-d1b5-4113-9d53-007089405527-kube-api-access-pjd68\") pod \"kuadrant-operator-controller-manager-55467d9f7-6ht6g\" (UID: \"bc930599-d1b5-4113-9d53-007089405527\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:24.441014 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.440993 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc930599-d1b5-4113-9d53-007089405527-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-6ht6g\" (UID: \"bc930599-d1b5-4113-9d53-007089405527\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:24.449740 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.449710 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjd68\" (UniqueName: \"kubernetes.io/projected/bc930599-d1b5-4113-9d53-007089405527-kube-api-access-pjd68\") pod \"kuadrant-operator-controller-manager-55467d9f7-6ht6g\" (UID: \"bc930599-d1b5-4113-9d53-007089405527\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:24.543007 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.542913 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:24.622549 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.622518 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g"] Jul 21 06:25:24.628679 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.628265 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g"] Jul 21 06:25:24.641194 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.641161 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f"] Jul 21 06:25:24.641461 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.641432 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" podUID="165a355e-98c4-4485-a804-709cfb0205ba" containerName="manager" containerID="cri-o://a82a5d2ba023fb8199dc0e335d0ae59734a23b8cf481b25b4e63bf761b4a0835" gracePeriod=2 Jul 21 06:25:24.653047 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.652840 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f"] Jul 21 06:25:24.657933 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:24.657091 2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"165a355e-98c4-4485-a804-709cfb0205ba\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\": pods \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:24.667547 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.667523 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj"] Jul 21 06:25:24.668088 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.668074 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="165a355e-98c4-4485-a804-709cfb0205ba" containerName="manager" Jul 21 06:25:24.668128 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.668092 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="165a355e-98c4-4485-a804-709cfb0205ba" containerName="manager" Jul 21 06:25:24.668283 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.668270 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="165a355e-98c4-4485-a804-709cfb0205ba" containerName="manager" Jul 21 06:25:24.671646 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.671627 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" Jul 21 06:25:24.674203 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:24.674176 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" is forbidden: User \"system:node:ip-10-0-140-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-159.ec2.internal' and this object" podUID="165a355e-98c4-4485-a804-709cfb0205ba" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:24.684249 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.684166 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj"] Jul 21 06:25:24.744585 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.744549 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5h76\" (UniqueName: \"kubernetes.io/projected/a3f99cef-cd2a-4891-bdda-0c19d9bc5222-kube-api-access-d5h76\") pod \"kuadrant-operator-controller-manager-8684c9ff49-56rfj\" (UID: \"a3f99cef-cd2a-4891-bdda-0c19d9bc5222\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" Jul 21 06:25:24.744776 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.744603 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3f99cef-cd2a-4891-bdda-0c19d9bc5222-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-56rfj\" (UID: \"a3f99cef-cd2a-4891-bdda-0c19d9bc5222\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" Jul 21 06:25:24.821518 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:24.821475 2563 log.go:32] "RunPodSandbox from runtime service failed" err=< Jul 21 06:25:24.821518 ip-10-0-140-159 kubenswrapper[2563]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-6ht6g_kuadrant-system_bc930599-d1b5-4113-9d53-007089405527_0(5e2bcd4c39e9bb921d20828a89d8f2b0f1dbd0e8d1424a229da33aaf35120ef6): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-6ht6g to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"5e2bcd4c39e9bb921d20828a89d8f2b0f1dbd0e8d1424a229da33aaf35120ef6" Netns:"/var/run/netns/bd76a3f7-e21c-4a00-be76-f72f2219c0c9" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-6ht6g;K8S_POD_INFRA_CONTAINER_ID=5e2bcd4c39e9bb921d20828a89d8f2b0f1dbd0e8d1424a229da33aaf35120ef6;K8S_POD_UID=bc930599-d1b5-4113-9d53-007089405527" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g/bc930599-d1b5-4113-9d53-007089405527:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g 5e2bcd4c39e9bb921d20828a89d8f2b0f1dbd0e8d1424a229da33aaf35120ef6 network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g 5e2bcd4c39e9bb921d20828a89d8f2b0f1dbd0e8d1424a229da33aaf35120ef6 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:86:00:2b [10.134.0.43/23] Jul 21 06:25:24.821518 ip-10-0-140-159 kubenswrapper[2563]: ' Jul 21 06:25:24.821518 ip-10-0-140-159 kubenswrapper[2563]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 21 06:25:24.821518 ip-10-0-140-159 kubenswrapper[2563]: > Jul 21 06:25:24.822059 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:24.821553 2563 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=< Jul 21 06:25:24.822059 ip-10-0-140-159 kubenswrapper[2563]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-6ht6g_kuadrant-system_bc930599-d1b5-4113-9d53-007089405527_0(5e2bcd4c39e9bb921d20828a89d8f2b0f1dbd0e8d1424a229da33aaf35120ef6): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-6ht6g to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"5e2bcd4c39e9bb921d20828a89d8f2b0f1dbd0e8d1424a229da33aaf35120ef6" Netns:"/var/run/netns/bd76a3f7-e21c-4a00-be76-f72f2219c0c9" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-6ht6g;K8S_POD_INFRA_CONTAINER_ID=5e2bcd4c39e9bb921d20828a89d8f2b0f1dbd0e8d1424a229da33aaf35120ef6;K8S_POD_UID=bc930599-d1b5-4113-9d53-007089405527" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g/bc930599-d1b5-4113-9d53-007089405527:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g 5e2bcd4c39e9bb921d20828a89d8f2b0f1dbd0e8d1424a229da33aaf35120ef6 network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g 5e2bcd4c39e9bb921d20828a89d8f2b0f1dbd0e8d1424a229da33aaf35120ef6 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:86:00:2b [10.134.0.43/23] Jul 21 06:25:24.822059 ip-10-0-140-159 kubenswrapper[2563]: ' Jul 21 06:25:24.822059 ip-10-0-140-159 kubenswrapper[2563]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 21 06:25:24.822059 ip-10-0-140-159 kubenswrapper[2563]: > pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:24.846153 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.846120 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5h76\" (UniqueName: \"kubernetes.io/projected/a3f99cef-cd2a-4891-bdda-0c19d9bc5222-kube-api-access-d5h76\") pod \"kuadrant-operator-controller-manager-8684c9ff49-56rfj\" (UID: \"a3f99cef-cd2a-4891-bdda-0c19d9bc5222\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" Jul 21 06:25:24.846344 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.846173 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3f99cef-cd2a-4891-bdda-0c19d9bc5222-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-56rfj\" (UID: \"a3f99cef-cd2a-4891-bdda-0c19d9bc5222\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" Jul 21 06:25:24.846527 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.846510 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3f99cef-cd2a-4891-bdda-0c19d9bc5222-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-56rfj\" (UID: \"a3f99cef-cd2a-4891-bdda-0c19d9bc5222\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" Jul 21 06:25:24.857870 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.857846 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5h76\" (UniqueName: \"kubernetes.io/projected/a3f99cef-cd2a-4891-bdda-0c19d9bc5222-kube-api-access-d5h76\") pod \"kuadrant-operator-controller-manager-8684c9ff49-56rfj\" (UID: \"a3f99cef-cd2a-4891-bdda-0c19d9bc5222\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" Jul 21 06:25:24.874076 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.874057 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:24.876767 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:24.876744 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" is forbidden: User \"system:node:ip-10-0-140-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-159.ec2.internal' and this object" podUID="165a355e-98c4-4485-a804-709cfb0205ba" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:24.878562 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.878543 2563 generic.go:356] "Generic (PLEG): container finished" podID="165a355e-98c4-4485-a804-709cfb0205ba" containerID="a82a5d2ba023fb8199dc0e335d0ae59734a23b8cf481b25b4e63bf761b4a0835" exitCode=0 Jul 21 06:25:24.878657 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.878583 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:24.878657 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.878646 2563 scope.go:117] "RemoveContainer" containerID="a82a5d2ba023fb8199dc0e335d0ae59734a23b8cf481b25b4e63bf761b4a0835" Jul 21 06:25:24.878749 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.878709 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:24.881341 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:24.881320 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" is forbidden: User \"system:node:ip-10-0-140-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-159.ec2.internal' and this object" podUID="165a355e-98c4-4485-a804-709cfb0205ba" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:24.883786 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:24.883761 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-6ht6g\" is forbidden: User \"system:node:ip-10-0-140-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-159.ec2.internal' and this object" podUID="bc930599-d1b5-4113-9d53-007089405527" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:24.885158 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.885138 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:24.886093 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:24.886069 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" is forbidden: User \"system:node:ip-10-0-140-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-159.ec2.internal' and this object" podUID="165a355e-98c4-4485-a804-709cfb0205ba" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:24.887490 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.887476 2563 scope.go:117] "RemoveContainer" containerID="a82a5d2ba023fb8199dc0e335d0ae59734a23b8cf481b25b4e63bf761b4a0835" Jul 21 06:25:24.887717 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:24.887699 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a82a5d2ba023fb8199dc0e335d0ae59734a23b8cf481b25b4e63bf761b4a0835\": container with ID starting with a82a5d2ba023fb8199dc0e335d0ae59734a23b8cf481b25b4e63bf761b4a0835 not found: ID does not exist" containerID="a82a5d2ba023fb8199dc0e335d0ae59734a23b8cf481b25b4e63bf761b4a0835" Jul 21 06:25:24.887765 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.887725 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a82a5d2ba023fb8199dc0e335d0ae59734a23b8cf481b25b4e63bf761b4a0835"} err="failed to get container status \"a82a5d2ba023fb8199dc0e335d0ae59734a23b8cf481b25b4e63bf761b4a0835\": rpc error: code = NotFound desc = could not find container \"a82a5d2ba023fb8199dc0e335d0ae59734a23b8cf481b25b4e63bf761b4a0835\": container with ID starting with a82a5d2ba023fb8199dc0e335d0ae59734a23b8cf481b25b4e63bf761b4a0835 not found: ID does not exist" Jul 21 06:25:24.888533 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:24.888515 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-6ht6g\" is forbidden: User \"system:node:ip-10-0-140-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-159.ec2.internal' and this object" podUID="bc930599-d1b5-4113-9d53-007089405527" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:24.890728 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:24.890710 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" is forbidden: User \"system:node:ip-10-0-140-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-159.ec2.internal' and this object" podUID="165a355e-98c4-4485-a804-709cfb0205ba" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:24.947243 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.947180 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcwq9\" (UniqueName: \"kubernetes.io/projected/165a355e-98c4-4485-a804-709cfb0205ba-kube-api-access-vcwq9\") pod \"165a355e-98c4-4485-a804-709cfb0205ba\" (UID: \"165a355e-98c4-4485-a804-709cfb0205ba\") " Jul 21 06:25:24.947407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.947258 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjd68\" (UniqueName: \"kubernetes.io/projected/bc930599-d1b5-4113-9d53-007089405527-kube-api-access-pjd68\") pod \"bc930599-d1b5-4113-9d53-007089405527\" (UID: \"bc930599-d1b5-4113-9d53-007089405527\") " Jul 21 06:25:24.947407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.947284 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/165a355e-98c4-4485-a804-709cfb0205ba-extensions-socket-volume\") pod \"165a355e-98c4-4485-a804-709cfb0205ba\" (UID: \"165a355e-98c4-4485-a804-709cfb0205ba\") " Jul 21 06:25:24.947407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.947337 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc930599-d1b5-4113-9d53-007089405527-extensions-socket-volume\") pod \"bc930599-d1b5-4113-9d53-007089405527\" (UID: \"bc930599-d1b5-4113-9d53-007089405527\") " Jul 21 06:25:24.947632 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.947610 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc930599-d1b5-4113-9d53-007089405527-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bc930599-d1b5-4113-9d53-007089405527" (UID: "bc930599-d1b5-4113-9d53-007089405527"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:25:24.947819 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.947796 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165a355e-98c4-4485-a804-709cfb0205ba-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "165a355e-98c4-4485-a804-709cfb0205ba" (UID: "165a355e-98c4-4485-a804-709cfb0205ba"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:25:24.949478 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.949456 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/165a355e-98c4-4485-a804-709cfb0205ba-kube-api-access-vcwq9" (OuterVolumeSpecName: "kube-api-access-vcwq9") pod "165a355e-98c4-4485-a804-709cfb0205ba" (UID: "165a355e-98c4-4485-a804-709cfb0205ba"). InnerVolumeSpecName "kube-api-access-vcwq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:25:24.949478 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:24.949465 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc930599-d1b5-4113-9d53-007089405527-kube-api-access-pjd68" (OuterVolumeSpecName: "kube-api-access-pjd68") pod "bc930599-d1b5-4113-9d53-007089405527" (UID: "bc930599-d1b5-4113-9d53-007089405527"). InnerVolumeSpecName "kube-api-access-pjd68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:25:25.021100 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:25.021051 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" Jul 21 06:25:25.048374 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:25.048344 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcwq9\" (UniqueName: \"kubernetes.io/projected/165a355e-98c4-4485-a804-709cfb0205ba-kube-api-access-vcwq9\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:25:25.048374 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:25.048371 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjd68\" (UniqueName: \"kubernetes.io/projected/bc930599-d1b5-4113-9d53-007089405527-kube-api-access-pjd68\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:25:25.048542 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:25.048382 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/165a355e-98c4-4485-a804-709cfb0205ba-extensions-socket-volume\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:25:25.048542 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:25.048390 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bc930599-d1b5-4113-9d53-007089405527-extensions-socket-volume\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:25:25.164337 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:25.164310 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj"] Jul 21 06:25:25.165586 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:25:25.165549 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f99cef_cd2a_4891_bdda_0c19d9bc5222.slice/crio-5a89f7d56b39574ef51e027d42337e0a46d809bf62d7cda48c444dd8d92968ab WatchSource:0}: Error finding container 5a89f7d56b39574ef51e027d42337e0a46d809bf62d7cda48c444dd8d92968ab: Status 404 returned error can't find the container with id 5a89f7d56b39574ef51e027d42337e0a46d809bf62d7cda48c444dd8d92968ab Jul 21 06:25:25.212858 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:25.212828 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-6ht6g\" is forbidden: User \"system:node:ip-10-0-140-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-159.ec2.internal' and this object" podUID="bc930599-d1b5-4113-9d53-007089405527" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:25.215231 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:25.215184 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" is forbidden: User \"system:node:ip-10-0-140-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-159.ec2.internal' and this object" podUID="165a355e-98c4-4485-a804-709cfb0205ba" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:25.885950 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:25.885911 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" event={"ID":"a3f99cef-cd2a-4891-bdda-0c19d9bc5222","Type":"ContainerStarted","Data":"109fe1037b328710b80e9c0b6fa439da277090f7a769951cc4ec20dce348c161"} Jul 21 06:25:25.886588 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:25.885974 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" event={"ID":"a3f99cef-cd2a-4891-bdda-0c19d9bc5222","Type":"ContainerStarted","Data":"5a89f7d56b39574ef51e027d42337e0a46d809bf62d7cda48c444dd8d92968ab"} Jul 21 06:25:25.886588 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:25.886105 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" Jul 21 06:25:25.887292 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:25.887277 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:25.905471 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:25.905413 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" podStartSLOduration=1.905395937 podStartE2EDuration="1.905395937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:25:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:25:25.904344773 +0000 UTC m=+455.849983430" watchObservedRunningTime="2026-07-21 06:25:25.905395937 +0000 UTC m=+455.851034595" Jul 21 06:25:25.906708 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:25.906683 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-6ht6g\" is forbidden: User \"system:node:ip-10-0-140-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-159.ec2.internal' and this object" podUID="bc930599-d1b5-4113-9d53-007089405527" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:25.908964 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:25.908939 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" is forbidden: User \"system:node:ip-10-0-140-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-159.ec2.internal' and this object" podUID="165a355e-98c4-4485-a804-709cfb0205ba" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:25.911454 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:25.911431 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-6ht6g\" is forbidden: User \"system:node:ip-10-0-140-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-159.ec2.internal' and this object" podUID="bc930599-d1b5-4113-9d53-007089405527" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-6ht6g" Jul 21 06:25:25.913896 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:25.913859 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6mv6f\" is forbidden: User \"system:node:ip-10-0-140-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-159.ec2.internal' and this object" podUID="165a355e-98c4-4485-a804-709cfb0205ba" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6mv6f" Jul 21 06:25:26.486506 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:26.486473 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="165a355e-98c4-4485-a804-709cfb0205ba" path="/var/lib/kubelet/pods/165a355e-98c4-4485-a804-709cfb0205ba/volumes" Jul 21 06:25:26.486812 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:26.486799 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc930599-d1b5-4113-9d53-007089405527" path="/var/lib/kubelet/pods/bc930599-d1b5-4113-9d53-007089405527/volumes" Jul 21 06:25:28.663103 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.663065 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-855fc695dd-frncg" podUID="9e657d37-915f-49b5-bd92-13c59b0a54b2" containerName="console" containerID="cri-o://ddb412e26a22cc6a4a38c6516d87296703a2ca3dee1049660249f71f29a5f46d" gracePeriod=15 Jul 21 06:25:28.906154 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.906126 2563 generic.go:356] "Generic (PLEG): container finished" podID="9e657d37-915f-49b5-bd92-13c59b0a54b2" containerID="ddb412e26a22cc6a4a38c6516d87296703a2ca3dee1049660249f71f29a5f46d" exitCode=0 Jul 21 06:25:28.906301 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.906208 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855fc695dd-frncg" event={"ID":"9e657d37-915f-49b5-bd92-13c59b0a54b2","Type":"ContainerDied","Data":"ddb412e26a22cc6a4a38c6516d87296703a2ca3dee1049660249f71f29a5f46d"} Jul 21 06:25:28.921683 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.921621 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:25:28.984476 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.984445 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-serving-cert\") pod \"9e657d37-915f-49b5-bd92-13c59b0a54b2\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " Jul 21 06:25:28.984476 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.984480 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-oauth-config\") pod \"9e657d37-915f-49b5-bd92-13c59b0a54b2\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " Jul 21 06:25:28.984705 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.984521 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-oauth-serving-cert\") pod \"9e657d37-915f-49b5-bd92-13c59b0a54b2\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " Jul 21 06:25:28.984705 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.984639 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-trusted-ca-bundle\") pod \"9e657d37-915f-49b5-bd92-13c59b0a54b2\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " Jul 21 06:25:28.984705 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.984690 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-config\") pod \"9e657d37-915f-49b5-bd92-13c59b0a54b2\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " Jul 21 06:25:28.984862 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.984732 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgp7m\" (UniqueName: \"kubernetes.io/projected/9e657d37-915f-49b5-bd92-13c59b0a54b2-kube-api-access-tgp7m\") pod \"9e657d37-915f-49b5-bd92-13c59b0a54b2\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " Jul 21 06:25:28.984862 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.984776 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-service-ca\") pod \"9e657d37-915f-49b5-bd92-13c59b0a54b2\" (UID: \"9e657d37-915f-49b5-bd92-13c59b0a54b2\") " Jul 21 06:25:28.985385 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.985326 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9e657d37-915f-49b5-bd92-13c59b0a54b2" (UID: "9e657d37-915f-49b5-bd92-13c59b0a54b2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:25:28.985707 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.985616 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9e657d37-915f-49b5-bd92-13c59b0a54b2" (UID: "9e657d37-915f-49b5-bd92-13c59b0a54b2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:25:28.985807 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.985697 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-service-ca" (OuterVolumeSpecName: "service-ca") pod "9e657d37-915f-49b5-bd92-13c59b0a54b2" (UID: "9e657d37-915f-49b5-bd92-13c59b0a54b2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:25:28.987718 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.986071 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-config" (OuterVolumeSpecName: "console-config") pod "9e657d37-915f-49b5-bd92-13c59b0a54b2" (UID: "9e657d37-915f-49b5-bd92-13c59b0a54b2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:25:28.989273 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.987887 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9e657d37-915f-49b5-bd92-13c59b0a54b2" (UID: "9e657d37-915f-49b5-bd92-13c59b0a54b2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:25:28.992858 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.989392 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e657d37-915f-49b5-bd92-13c59b0a54b2-kube-api-access-tgp7m" (OuterVolumeSpecName: "kube-api-access-tgp7m") pod "9e657d37-915f-49b5-bd92-13c59b0a54b2" (UID: "9e657d37-915f-49b5-bd92-13c59b0a54b2"). InnerVolumeSpecName "kube-api-access-tgp7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:25:28.992949 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:28.992893 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9e657d37-915f-49b5-bd92-13c59b0a54b2" (UID: "9e657d37-915f-49b5-bd92-13c59b0a54b2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 06:25:29.087854 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:29.087811 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-serving-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:25:29.087854 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:29.087843 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-oauth-config\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:25:29.087854 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:29.087857 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-oauth-serving-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:25:29.088087 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:29.087870 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-trusted-ca-bundle\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:25:29.088087 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:29.087882 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-console-config\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:25:29.088087 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:29.087895 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgp7m\" (UniqueName: \"kubernetes.io/projected/9e657d37-915f-49b5-bd92-13c59b0a54b2-kube-api-access-tgp7m\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:25:29.088087 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:29.087906 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e657d37-915f-49b5-bd92-13c59b0a54b2-service-ca\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:25:29.913588 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:29.913554 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-855fc695dd-frncg" event={"ID":"9e657d37-915f-49b5-bd92-13c59b0a54b2","Type":"ContainerDied","Data":"cecf8aa0d05c722a83e879a6bc1e0e5bc01c7005f37940c30afd1739a2ffc669"} Jul 21 06:25:29.914048 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:29.913612 2563 scope.go:117] "RemoveContainer" containerID="ddb412e26a22cc6a4a38c6516d87296703a2ca3dee1049660249f71f29a5f46d" Jul 21 06:25:29.914048 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:29.913613 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-855fc695dd-frncg" Jul 21 06:25:29.935600 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:29.935571 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-855fc695dd-frncg"] Jul 21 06:25:29.940935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:29.940914 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-855fc695dd-frncg"] Jul 21 06:25:30.486103 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:30.486070 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e657d37-915f-49b5-bd92-13c59b0a54b2" path="/var/lib/kubelet/pods/9e657d37-915f-49b5-bd92-13c59b0a54b2/volumes" Jul 21 06:25:36.895403 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:36.895375 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" Jul 21 06:25:51.856868 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:51.856832 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj"] Jul 21 06:25:51.857364 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:51.857100 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" podUID="a3f99cef-cd2a-4891-bdda-0c19d9bc5222" containerName="manager" containerID="cri-o://109fe1037b328710b80e9c0b6fa439da277090f7a769951cc4ec20dce348c161" gracePeriod=10 Jul 21 06:25:51.882789 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:25:51.882753 2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f99cef_cd2a_4891_bdda_0c19d9bc5222.slice/crio-109fe1037b328710b80e9c0b6fa439da277090f7a769951cc4ec20dce348c161.scope\": RecentStats: unable to find data in memory cache]" Jul 21 06:25:52.037827 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.037801 2563 generic.go:356] "Generic (PLEG): container finished" podID="a3f99cef-cd2a-4891-bdda-0c19d9bc5222" containerID="109fe1037b328710b80e9c0b6fa439da277090f7a769951cc4ec20dce348c161" exitCode=0 Jul 21 06:25:52.037973 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.037879 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" event={"ID":"a3f99cef-cd2a-4891-bdda-0c19d9bc5222","Type":"ContainerDied","Data":"109fe1037b328710b80e9c0b6fa439da277090f7a769951cc4ec20dce348c161"} Jul 21 06:25:52.100862 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.100833 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9"] Jul 21 06:25:52.101203 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.101191 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e657d37-915f-49b5-bd92-13c59b0a54b2" containerName="console" Jul 21 06:25:52.101275 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.101205 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e657d37-915f-49b5-bd92-13c59b0a54b2" containerName="console" Jul 21 06:25:52.101311 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.101292 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e657d37-915f-49b5-bd92-13c59b0a54b2" containerName="console" Jul 21 06:25:52.104415 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.104399 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" Jul 21 06:25:52.108704 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.108651 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" Jul 21 06:25:52.116696 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.116670 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9"] Jul 21 06:25:52.293470 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.293439 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5h76\" (UniqueName: \"kubernetes.io/projected/a3f99cef-cd2a-4891-bdda-0c19d9bc5222-kube-api-access-d5h76\") pod \"a3f99cef-cd2a-4891-bdda-0c19d9bc5222\" (UID: \"a3f99cef-cd2a-4891-bdda-0c19d9bc5222\") " Jul 21 06:25:52.293650 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.293588 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3f99cef-cd2a-4891-bdda-0c19d9bc5222-extensions-socket-volume\") pod \"a3f99cef-cd2a-4891-bdda-0c19d9bc5222\" (UID: \"a3f99cef-cd2a-4891-bdda-0c19d9bc5222\") " Jul 21 06:25:52.293798 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.293771 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppwb8\" (UniqueName: \"kubernetes.io/projected/79f0cd21-db64-49a3-8687-fe920d87fd88-kube-api-access-ppwb8\") pod \"kuadrant-operator-controller-manager-86878c769f-fp6z9\" (UID: \"79f0cd21-db64-49a3-8687-fe920d87fd88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" Jul 21 06:25:52.293874 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.293858 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/79f0cd21-db64-49a3-8687-fe920d87fd88-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fp6z9\" (UID: \"79f0cd21-db64-49a3-8687-fe920d87fd88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" Jul 21 06:25:52.293938 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.293907 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3f99cef-cd2a-4891-bdda-0c19d9bc5222-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a3f99cef-cd2a-4891-bdda-0c19d9bc5222" (UID: "a3f99cef-cd2a-4891-bdda-0c19d9bc5222"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:25:52.295664 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.295644 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3f99cef-cd2a-4891-bdda-0c19d9bc5222-kube-api-access-d5h76" (OuterVolumeSpecName: "kube-api-access-d5h76") pod "a3f99cef-cd2a-4891-bdda-0c19d9bc5222" (UID: "a3f99cef-cd2a-4891-bdda-0c19d9bc5222"). InnerVolumeSpecName "kube-api-access-d5h76". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:25:52.395160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.395085 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/79f0cd21-db64-49a3-8687-fe920d87fd88-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fp6z9\" (UID: \"79f0cd21-db64-49a3-8687-fe920d87fd88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" Jul 21 06:25:52.395160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.395160 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppwb8\" (UniqueName: \"kubernetes.io/projected/79f0cd21-db64-49a3-8687-fe920d87fd88-kube-api-access-ppwb8\") pod \"kuadrant-operator-controller-manager-86878c769f-fp6z9\" (UID: \"79f0cd21-db64-49a3-8687-fe920d87fd88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" Jul 21 06:25:52.395403 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.395195 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3f99cef-cd2a-4891-bdda-0c19d9bc5222-extensions-socket-volume\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:25:52.395403 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.395204 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5h76\" (UniqueName: \"kubernetes.io/projected/a3f99cef-cd2a-4891-bdda-0c19d9bc5222-kube-api-access-d5h76\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:25:52.395561 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.395540 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/79f0cd21-db64-49a3-8687-fe920d87fd88-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fp6z9\" (UID: \"79f0cd21-db64-49a3-8687-fe920d87fd88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" Jul 21 06:25:52.403616 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.403589 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppwb8\" (UniqueName: \"kubernetes.io/projected/79f0cd21-db64-49a3-8687-fe920d87fd88-kube-api-access-ppwb8\") pod \"kuadrant-operator-controller-manager-86878c769f-fp6z9\" (UID: \"79f0cd21-db64-49a3-8687-fe920d87fd88\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" Jul 21 06:25:52.414570 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.414547 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" Jul 21 06:25:52.549114 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:52.549089 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9"] Jul 21 06:25:52.551020 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:25:52.550983 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f0cd21_db64_49a3_8687_fe920d87fd88.slice/crio-df9a5c57763d3e34255ae09dc5479acc3f483b12d662448bb6726a101affd658 WatchSource:0}: Error finding container df9a5c57763d3e34255ae09dc5479acc3f483b12d662448bb6726a101affd658: Status 404 returned error can't find the container with id df9a5c57763d3e34255ae09dc5479acc3f483b12d662448bb6726a101affd658 Jul 21 06:25:53.044550 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:53.044517 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" event={"ID":"79f0cd21-db64-49a3-8687-fe920d87fd88","Type":"ContainerStarted","Data":"c60269065e334e26775e55faedd1f49fdcbd5ee188bd2f35269e07b7d681c1ad"} Jul 21 06:25:53.045003 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:53.044568 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" event={"ID":"79f0cd21-db64-49a3-8687-fe920d87fd88","Type":"ContainerStarted","Data":"df9a5c57763d3e34255ae09dc5479acc3f483b12d662448bb6726a101affd658"} Jul 21 06:25:53.045003 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:53.044617 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" Jul 21 06:25:53.045868 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:53.045849 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" Jul 21 06:25:53.045978 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:53.045893 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj" event={"ID":"a3f99cef-cd2a-4891-bdda-0c19d9bc5222","Type":"ContainerDied","Data":"5a89f7d56b39574ef51e027d42337e0a46d809bf62d7cda48c444dd8d92968ab"} Jul 21 06:25:53.045978 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:53.045943 2563 scope.go:117] "RemoveContainer" containerID="109fe1037b328710b80e9c0b6fa439da277090f7a769951cc4ec20dce348c161" Jul 21 06:25:53.066144 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:53.066109 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" podStartSLOduration=1.066097091 podStartE2EDuration="1.066097091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:25:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:25:53.064029611 +0000 UTC m=+483.009668269" watchObservedRunningTime="2026-07-21 06:25:53.066097091 +0000 UTC m=+483.011735747" Jul 21 06:25:53.078564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:53.078543 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj"] Jul 21 06:25:53.082341 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:53.082322 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-56rfj"] Jul 21 06:25:54.487141 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:25:54.487112 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3f99cef-cd2a-4891-bdda-0c19d9bc5222" path="/var/lib/kubelet/pods/a3f99cef-cd2a-4891-bdda-0c19d9bc5222/volumes" Jul 21 06:26:04.054271 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:04.054198 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" Jul 21 06:26:08.183337 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.183305 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj"] Jul 21 06:26:08.183707 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.183695 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3f99cef-cd2a-4891-bdda-0c19d9bc5222" containerName="manager" Jul 21 06:26:08.183746 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.183709 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3f99cef-cd2a-4891-bdda-0c19d9bc5222" containerName="manager" Jul 21 06:26:08.183791 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.183779 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3f99cef-cd2a-4891-bdda-0c19d9bc5222" containerName="manager" Jul 21 06:26:08.188296 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.188276 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.191320 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.191301 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-7zjjb\"" Jul 21 06:26:08.198634 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.198613 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj"] Jul 21 06:26:08.233698 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.233668 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.233881 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.233707 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c492a72-9f73-46e1-b8df-4e771849cccf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.233881 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.233738 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djmdb\" (UniqueName: \"kubernetes.io/projected/1c492a72-9f73-46e1-b8df-4e771849cccf-kube-api-access-djmdb\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.233881 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.233848 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.234070 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.233880 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c492a72-9f73-46e1-b8df-4e771849cccf-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.234070 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.233943 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.234070 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.233976 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.234070 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.234018 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.234281 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.234087 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c492a72-9f73-46e1-b8df-4e771849cccf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.335369 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.335337 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.335553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.335388 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c492a72-9f73-46e1-b8df-4e771849cccf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.335553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.335444 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.335553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.335475 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c492a72-9f73-46e1-b8df-4e771849cccf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.335553 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.335504 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djmdb\" (UniqueName: \"kubernetes.io/projected/1c492a72-9f73-46e1-b8df-4e771849cccf-kube-api-access-djmdb\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.335778 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.335555 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.335778 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.335578 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c492a72-9f73-46e1-b8df-4e771849cccf-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.335778 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.335620 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.335778 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.335651 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.335991 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.335785 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.335991 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.335816 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.335991 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.335890 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.336136 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.336047 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.336185 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.336152 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c492a72-9f73-46e1-b8df-4e771849cccf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.338041 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.338014 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c492a72-9f73-46e1-b8df-4e771849cccf-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.338152 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.338084 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c492a72-9f73-46e1-b8df-4e771849cccf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.346038 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.346016 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djmdb\" (UniqueName: \"kubernetes.io/projected/1c492a72-9f73-46e1-b8df-4e771849cccf-kube-api-access-djmdb\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.346148 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.346091 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c492a72-9f73-46e1-b8df-4e771849cccf-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-9k8mj\" (UID: \"1c492a72-9f73-46e1-b8df-4e771849cccf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.499316 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.499237 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:08.638121 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.638086 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj"] Jul 21 06:26:08.639451 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:26:08.639424 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c492a72_9f73_46e1_b8df_4e771849cccf.slice/crio-bcf6319c306b527b3372665758443be72ffdc2923e1da9738dfdd8777e9c2684 WatchSource:0}: Error finding container bcf6319c306b527b3372665758443be72ffdc2923e1da9738dfdd8777e9c2684: Status 404 returned error can't find the container with id bcf6319c306b527b3372665758443be72ffdc2923e1da9738dfdd8777e9c2684 Jul 21 06:26:08.641535 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.641503 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:26:08.641657 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.641565 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:26:08.641657 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:08.641596 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:26:09.137860 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:09.137816 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" event={"ID":"1c492a72-9f73-46e1-b8df-4e771849cccf","Type":"ContainerStarted","Data":"a7ff276fb4e5232efa973225eccc1582c435671bbf9be08fe8f4e351fd386611"} Jul 21 06:26:09.138048 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:09.137880 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" event={"ID":"1c492a72-9f73-46e1-b8df-4e771849cccf","Type":"ContainerStarted","Data":"bcf6319c306b527b3372665758443be72ffdc2923e1da9738dfdd8777e9c2684"} Jul 21 06:26:09.169994 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:09.169951 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" podStartSLOduration=1.169935996 podStartE2EDuration="1.169935996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:26:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:26:09.16696087 +0000 UTC m=+499.112599529" watchObservedRunningTime="2026-07-21 06:26:09.169935996 +0000 UTC m=+499.115574652" Jul 21 06:26:09.499631 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:09.499542 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:09.504325 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:09.504304 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:10.143762 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:10.143728 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:26:10.144692 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:26:10.144671 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:27:31.343600 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:31.343562 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-mpdhh"] Jul 21 06:27:31.347032 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:31.347004 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" Jul 21 06:27:31.350717 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:31.350695 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 21 06:27:31.350859 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:31.350845 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-b6vhf\"" Jul 21 06:27:31.370354 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:31.370325 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-mpdhh"] Jul 21 06:27:31.476863 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:31.476828 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4qbs\" (UniqueName: \"kubernetes.io/projected/4dee74c0-b8f6-40ac-ae92-048eef2fa77f-kube-api-access-s4qbs\") pod \"payload-pre-processing-7677b774bb-mpdhh\" (UID: \"4dee74c0-b8f6-40ac-ae92-048eef2fa77f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" Jul 21 06:27:31.477069 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:31.476890 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dee74c0-b8f6-40ac-ae92-048eef2fa77f-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-mpdhh\" (UID: \"4dee74c0-b8f6-40ac-ae92-048eef2fa77f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" Jul 21 06:27:31.577710 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:31.577678 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4qbs\" (UniqueName: \"kubernetes.io/projected/4dee74c0-b8f6-40ac-ae92-048eef2fa77f-kube-api-access-s4qbs\") pod \"payload-pre-processing-7677b774bb-mpdhh\" (UID: \"4dee74c0-b8f6-40ac-ae92-048eef2fa77f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" Jul 21 06:27:31.577904 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:31.577734 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dee74c0-b8f6-40ac-ae92-048eef2fa77f-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-mpdhh\" (UID: \"4dee74c0-b8f6-40ac-ae92-048eef2fa77f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" Jul 21 06:27:31.578614 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:31.578586 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dee74c0-b8f6-40ac-ae92-048eef2fa77f-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-mpdhh\" (UID: \"4dee74c0-b8f6-40ac-ae92-048eef2fa77f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" Jul 21 06:27:31.585748 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:31.585726 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4qbs\" (UniqueName: \"kubernetes.io/projected/4dee74c0-b8f6-40ac-ae92-048eef2fa77f-kube-api-access-s4qbs\") pod \"payload-pre-processing-7677b774bb-mpdhh\" (UID: \"4dee74c0-b8f6-40ac-ae92-048eef2fa77f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" Jul 21 06:27:31.657747 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:31.657681 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" Jul 21 06:27:31.793870 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:31.793836 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-mpdhh"] Jul 21 06:27:31.795726 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:27:31.795689 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dee74c0_b8f6_40ac_ae92_048eef2fa77f.slice/crio-69335d196ce543705f4c8e1700aadaaa1fb1da9bb49590d86fa8179e5c737e56 WatchSource:0}: Error finding container 69335d196ce543705f4c8e1700aadaaa1fb1da9bb49590d86fa8179e5c737e56: Status 404 returned error can't find the container with id 69335d196ce543705f4c8e1700aadaaa1fb1da9bb49590d86fa8179e5c737e56 Jul 21 06:27:32.599237 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:32.599177 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" event={"ID":"4dee74c0-b8f6-40ac-ae92-048eef2fa77f","Type":"ContainerStarted","Data":"69335d196ce543705f4c8e1700aadaaa1fb1da9bb49590d86fa8179e5c737e56"} Jul 21 06:27:34.611497 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:34.611458 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" event={"ID":"4dee74c0-b8f6-40ac-ae92-048eef2fa77f","Type":"ContainerStarted","Data":"9f301b773017eaba9c2cc459916da9fd7795299022fd82bc18365fbd9c30b804"} Jul 21 06:27:34.611866 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:34.611520 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" Jul 21 06:27:34.629988 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:34.629946 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" podStartSLOduration=1.559682579 podStartE2EDuration="3.629931561s" totalImagesPullingTime="2.070248982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:27:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:27:34.628529566 +0000 UTC m=+584.574168224" watchObservedRunningTime="2026-07-21 06:27:34.629931561 +0000 UTC m=+584.575570219" Jul 21 06:27:39.642186 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:39.642144 2563 generic.go:356] "Generic (PLEG): container finished" podID="1c492a72-9f73-46e1-b8df-4e771849cccf" containerID="a7ff276fb4e5232efa973225eccc1582c435671bbf9be08fe8f4e351fd386611" exitCode=0 Jul 21 06:27:39.642716 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:39.642202 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" event={"ID":"1c492a72-9f73-46e1-b8df-4e771849cccf","Type":"ContainerDied","Data":"a7ff276fb4e5232efa973225eccc1582c435671bbf9be08fe8f4e351fd386611"} Jul 21 06:27:39.642716 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:39.642620 2563 scope.go:117] "RemoveContainer" containerID="a7ff276fb4e5232efa973225eccc1582c435671bbf9be08fe8f4e351fd386611" Jul 21 06:27:39.643619 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:39.643595 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:27:39.643683 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:39.643658 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:27:39.643740 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:39.643690 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:27:40.144257 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:40.144191 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:27:40.653426 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:40.653389 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" event={"ID":"1c492a72-9f73-46e1-b8df-4e771849cccf","Type":"ContainerStarted","Data":"9e8633f59b52c04ced02d5c04927e65ce40f6b1d7cce0e65bd9e0e32fe9c4b06"} Jul 21 06:27:41.499817 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:41.499785 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:27:41.500801 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:41.500777 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-9k8mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 21 06:27:41.500902 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:41.500824 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" podUID="1c492a72-9f73-46e1-b8df-4e771849cccf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 21 06:27:42.499750 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:42.499715 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-9k8mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 21 06:27:42.500186 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:42.499769 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" podUID="1c492a72-9f73-46e1-b8df-4e771849cccf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 21 06:27:43.500036 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:43.499999 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-9k8mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 21 06:27:43.500443 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:43.500055 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" podUID="1c492a72-9f73-46e1-b8df-4e771849cccf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 21 06:27:44.511025 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:44.510991 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:27:44.511466 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:44.511253 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:27:44.511896 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:44.511880 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-9k8mj" Jul 21 06:27:45.618648 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:45.618605 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" podUID="4dee74c0-b8f6-40ac-ae92-048eef2fa77f" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.47:9004: connect: connection refused" Jul 21 06:27:55.618418 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:55.618387 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-mpdhh" Jul 21 06:27:57.484298 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.484265 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-826h5"] Jul 21 06:27:57.488696 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.488676 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.493176 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.493151 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 06:27:57.493388 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.493158 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-r52vr\"" Jul 21 06:27:57.493388 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.493159 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 21 06:27:57.493388 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.493271 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 06:27:57.496879 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.496861 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-826h5"] Jul 21 06:27:57.608280 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.608248 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.608452 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.608295 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.608452 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.608323 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.608452 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.608343 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2nlp\" (UniqueName: \"kubernetes.io/projected/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-kube-api-access-w2nlp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.608452 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.608366 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.608612 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.608477 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.709769 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.709733 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.709979 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.709808 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.709979 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.709931 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.709979 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.709969 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2nlp\" (UniqueName: \"kubernetes.io/projected/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-kube-api-access-w2nlp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.710163 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.709995 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.710163 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.710070 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.710310 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.710180 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.710403 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.710382 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.710456 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.710437 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.712337 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.712318 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.712566 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.712547 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.717941 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.717918 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2nlp\" (UniqueName: \"kubernetes.io/projected/6be1d497-31f8-4579-b75c-ff2a57a6d1b0-kube-api-access-w2nlp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-826h5\" (UID: \"6be1d497-31f8-4579-b75c-ff2a57a6d1b0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.800449 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.800357 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:27:57.933228 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:57.933184 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-826h5"] Jul 21 06:27:57.934296 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:27:57.934261 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be1d497_31f8_4579_b75c_ff2a57a6d1b0.slice/crio-7817628066b12e5b594029e84ef8c6eda0dfac001a4e982b1c1f7912205b4770 WatchSource:0}: Error finding container 7817628066b12e5b594029e84ef8c6eda0dfac001a4e982b1c1f7912205b4770: Status 404 returned error can't find the container with id 7817628066b12e5b594029e84ef8c6eda0dfac001a4e982b1c1f7912205b4770 Jul 21 06:27:58.759620 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:58.759586 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" event={"ID":"6be1d497-31f8-4579-b75c-ff2a57a6d1b0","Type":"ContainerStarted","Data":"7817628066b12e5b594029e84ef8c6eda0dfac001a4e982b1c1f7912205b4770"} Jul 21 06:27:59.767939 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:59.767901 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" event={"ID":"6be1d497-31f8-4579-b75c-ff2a57a6d1b0","Type":"ContainerStarted","Data":"6c4dc1b0df9a0f4babe86a716c45a20a1926230f54e48cb9194b567d4d0aa46e"} Jul 21 06:27:59.786537 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:27:59.786494 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" podStartSLOduration=1.7622568250000001 podStartE2EDuration="2.786478956s" totalImagesPullingTime="1.024222131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:27:59.78520201 +0000 UTC m=+609.730840667" watchObservedRunningTime="2026-07-21 06:27:59.786478956 +0000 UTC m=+609.732117656" Jul 21 06:28:01.793820 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.793791 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw"] Jul 21 06:28:01.797356 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.797338 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.800250 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.800228 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 21 06:28:01.807570 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.807544 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw"] Jul 21 06:28:01.852103 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.852065 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dafddc54-e5bd-4a22-a334-c4b11c61dd27-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.852447 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.852420 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dafddc54-e5bd-4a22-a334-c4b11c61dd27-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.852564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.852539 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dafddc54-e5bd-4a22-a334-c4b11c61dd27-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.852656 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.852609 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtv7b\" (UniqueName: \"kubernetes.io/projected/dafddc54-e5bd-4a22-a334-c4b11c61dd27-kube-api-access-dtv7b\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.852656 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.852649 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dafddc54-e5bd-4a22-a334-c4b11c61dd27-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.852764 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.852680 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dafddc54-e5bd-4a22-a334-c4b11c61dd27-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.953151 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.953114 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtv7b\" (UniqueName: \"kubernetes.io/projected/dafddc54-e5bd-4a22-a334-c4b11c61dd27-kube-api-access-dtv7b\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.953151 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.953154 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dafddc54-e5bd-4a22-a334-c4b11c61dd27-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.953445 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.953182 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dafddc54-e5bd-4a22-a334-c4b11c61dd27-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.953445 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.953249 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dafddc54-e5bd-4a22-a334-c4b11c61dd27-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.953445 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.953270 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dafddc54-e5bd-4a22-a334-c4b11c61dd27-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.953445 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.953308 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dafddc54-e5bd-4a22-a334-c4b11c61dd27-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.953615 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.953587 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dafddc54-e5bd-4a22-a334-c4b11c61dd27-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.953651 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.953638 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dafddc54-e5bd-4a22-a334-c4b11c61dd27-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.953702 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.953679 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dafddc54-e5bd-4a22-a334-c4b11c61dd27-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.955658 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.955636 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dafddc54-e5bd-4a22-a334-c4b11c61dd27-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.955903 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.955886 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dafddc54-e5bd-4a22-a334-c4b11c61dd27-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:01.962712 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:01.962687 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtv7b\" (UniqueName: \"kubernetes.io/projected/dafddc54-e5bd-4a22-a334-c4b11c61dd27-kube-api-access-dtv7b\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lh8nw\" (UID: \"dafddc54-e5bd-4a22-a334-c4b11c61dd27\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:02.109501 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:02.109465 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:02.247193 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:02.247166 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw"] Jul 21 06:28:02.249590 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:28:02.249565 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddafddc54_e5bd_4a22_a334_c4b11c61dd27.slice/crio-34306c596dc697d369ae0940ba13597155d36fd76539477619134da45afb5dd1 WatchSource:0}: Error finding container 34306c596dc697d369ae0940ba13597155d36fd76539477619134da45afb5dd1: Status 404 returned error can't find the container with id 34306c596dc697d369ae0940ba13597155d36fd76539477619134da45afb5dd1 Jul 21 06:28:02.786733 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:02.786630 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" event={"ID":"dafddc54-e5bd-4a22-a334-c4b11c61dd27","Type":"ContainerStarted","Data":"ca7534e364d64c371e4139de5ceab8453d18ebe02cba8c16db4c331a92edcfa2"} Jul 21 06:28:02.786733 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:02.786694 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" event={"ID":"dafddc54-e5bd-4a22-a334-c4b11c61dd27","Type":"ContainerStarted","Data":"34306c596dc697d369ae0940ba13597155d36fd76539477619134da45afb5dd1"} Jul 21 06:28:02.812578 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:02.812531 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" podStartSLOduration=1.622085231 podStartE2EDuration="1.812517085s" totalImagesPullingTime="190.431854ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:28:02.810342992 +0000 UTC m=+612.755981650" watchObservedRunningTime="2026-07-21 06:28:02.812517085 +0000 UTC m=+612.758155785" Jul 21 06:28:07.801308 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:07.801277 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:28:07.801706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:07.801568 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:28:07.815051 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:07.815026 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:28:07.826176 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:07.826145 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-826h5" Jul 21 06:28:12.110598 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:12.110511 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:12.110598 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:12.110600 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:12.123024 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:12.123004 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:28:12.853282 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:28:12.853255 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lh8nw" Jul 21 06:30:00.140080 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:00.139999 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743590-59s7g"] Jul 21 06:30:00.143526 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:00.143510 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743590-59s7g" Jul 21 06:30:00.146681 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:00.146657 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jvhnq\"" Jul 21 06:30:00.150019 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:00.149993 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743590-59s7g"] Jul 21 06:30:00.154429 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:00.154412 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743590-59s7g" Jul 21 06:30:00.493158 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:00.493138 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743590-59s7g"] Jul 21 06:30:00.494760 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:30:00.494731 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41f0c3d_6fd5_454b_9f40_9c4abeb088fb.slice/crio-58749d717ab9db49e4ad77f61cab86b233cb96005146921fd73898be7b8875f0 WatchSource:0}: Error finding container 58749d717ab9db49e4ad77f61cab86b233cb96005146921fd73898be7b8875f0: Status 404 returned error can't find the container with id 58749d717ab9db49e4ad77f61cab86b233cb96005146921fd73898be7b8875f0 Jul 21 06:30:00.496544 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:00.496527 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 06:30:01.433069 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:01.433024 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743590-59s7g" event={"ID":"d41f0c3d-6fd5-454b-9f40-9c4abeb088fb","Type":"ContainerStarted","Data":"58749d717ab9db49e4ad77f61cab86b233cb96005146921fd73898be7b8875f0"} Jul 21 06:30:03.447941 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:03.447909 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743590-59s7g" event={"ID":"d41f0c3d-6fd5-454b-9f40-9c4abeb088fb","Type":"ContainerStarted","Data":"6c4a411387283f559dcf317d9462f20c54011e13cdffd38ca99c6046e3850754"} Jul 21 06:30:03.470858 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:03.470806 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743590-59s7g" podStartSLOduration=1.3890187410000001 podStartE2EDuration="3.47079283s" totalImagesPullingTime="2.081774089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:30:03.469690068 +0000 UTC m=+733.415328726" watchObservedRunningTime="2026-07-21 06:30:03.47079283 +0000 UTC m=+733.416431487" Jul 21 06:30:04.456203 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:04.456169 2563 generic.go:356] "Generic (PLEG): container finished" podID="d41f0c3d-6fd5-454b-9f40-9c4abeb088fb" containerID="6c4a411387283f559dcf317d9462f20c54011e13cdffd38ca99c6046e3850754" exitCode=0 Jul 21 06:30:04.456654 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:04.456252 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743590-59s7g" event={"ID":"d41f0c3d-6fd5-454b-9f40-9c4abeb088fb","Type":"ContainerDied","Data":"6c4a411387283f559dcf317d9462f20c54011e13cdffd38ca99c6046e3850754"} Jul 21 06:30:05.595698 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:05.595676 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743590-59s7g" Jul 21 06:30:06.468790 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:06.468768 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743590-59s7g" Jul 21 06:30:06.468790 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:06.468781 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743590-59s7g" event={"ID":"d41f0c3d-6fd5-454b-9f40-9c4abeb088fb","Type":"ContainerDied","Data":"58749d717ab9db49e4ad77f61cab86b233cb96005146921fd73898be7b8875f0"} Jul 21 06:30:06.469001 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:30:06.468821 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58749d717ab9db49e4ad77f61cab86b233cb96005146921fd73898be7b8875f0" Jul 21 06:36:31.973070 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:31.973033 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9"] Jul 21 06:36:31.973694 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:31.973333 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" podUID="79f0cd21-db64-49a3-8687-fe920d87fd88" containerName="manager" containerID="cri-o://c60269065e334e26775e55faedd1f49fdcbd5ee188bd2f35269e07b7d681c1ad" gracePeriod=10 Jul 21 06:36:34.052591 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:34.052561 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" podUID="79f0cd21-db64-49a3-8687-fe920d87fd88" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.45:8081/readyz\": dial tcp 10.134.0.45:8081: connect: connection refused" Jul 21 06:36:34.622126 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:34.622100 2563 generic.go:356] "Generic (PLEG): container finished" podID="79f0cd21-db64-49a3-8687-fe920d87fd88" containerID="c60269065e334e26775e55faedd1f49fdcbd5ee188bd2f35269e07b7d681c1ad" exitCode=0 Jul 21 06:36:34.622263 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:34.622139 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" event={"ID":"79f0cd21-db64-49a3-8687-fe920d87fd88","Type":"ContainerDied","Data":"c60269065e334e26775e55faedd1f49fdcbd5ee188bd2f35269e07b7d681c1ad"} Jul 21 06:36:34.622263 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:34.622174 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" event={"ID":"79f0cd21-db64-49a3-8687-fe920d87fd88","Type":"ContainerDied","Data":"df9a5c57763d3e34255ae09dc5479acc3f483b12d662448bb6726a101affd658"} Jul 21 06:36:34.622263 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:34.622191 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df9a5c57763d3e34255ae09dc5479acc3f483b12d662448bb6726a101affd658" Jul 21 06:36:34.623135 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:34.623118 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" Jul 21 06:36:34.753859 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:34.753769 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppwb8\" (UniqueName: \"kubernetes.io/projected/79f0cd21-db64-49a3-8687-fe920d87fd88-kube-api-access-ppwb8\") pod \"79f0cd21-db64-49a3-8687-fe920d87fd88\" (UID: \"79f0cd21-db64-49a3-8687-fe920d87fd88\") " Jul 21 06:36:34.754004 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:34.753866 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/79f0cd21-db64-49a3-8687-fe920d87fd88-extensions-socket-volume\") pod \"79f0cd21-db64-49a3-8687-fe920d87fd88\" (UID: \"79f0cd21-db64-49a3-8687-fe920d87fd88\") " Jul 21 06:36:34.754271 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:34.754211 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f0cd21-db64-49a3-8687-fe920d87fd88-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "79f0cd21-db64-49a3-8687-fe920d87fd88" (UID: "79f0cd21-db64-49a3-8687-fe920d87fd88"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:36:34.756021 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:34.755996 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79f0cd21-db64-49a3-8687-fe920d87fd88-kube-api-access-ppwb8" (OuterVolumeSpecName: "kube-api-access-ppwb8") pod "79f0cd21-db64-49a3-8687-fe920d87fd88" (UID: "79f0cd21-db64-49a3-8687-fe920d87fd88"). InnerVolumeSpecName "kube-api-access-ppwb8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:36:34.855504 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:34.855477 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppwb8\" (UniqueName: \"kubernetes.io/projected/79f0cd21-db64-49a3-8687-fe920d87fd88-kube-api-access-ppwb8\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:36:34.855504 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:34.855498 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/79f0cd21-db64-49a3-8687-fe920d87fd88-extensions-socket-volume\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:36:35.627322 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:35.627289 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9" Jul 21 06:36:35.649574 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:35.649548 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9"] Jul 21 06:36:35.653028 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:35.653005 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fp6z9"] Jul 21 06:36:36.487290 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:36.487258 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79f0cd21-db64-49a3-8687-fe920d87fd88" path="/var/lib/kubelet/pods/79f0cd21-db64-49a3-8687-fe920d87fd88/volumes" Jul 21 06:36:50.531362 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:36:50.531334 2563 scope.go:117] "RemoveContainer" containerID="c60269065e334e26775e55faedd1f49fdcbd5ee188bd2f35269e07b7d681c1ad" Jul 21 06:37:33.192904 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.192870 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx"] Jul 21 06:37:33.193400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.193288 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d41f0c3d-6fd5-454b-9f40-9c4abeb088fb" containerName="cleanup" Jul 21 06:37:33.193400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.193299 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d41f0c3d-6fd5-454b-9f40-9c4abeb088fb" containerName="cleanup" Jul 21 06:37:33.193400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.193310 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79f0cd21-db64-49a3-8687-fe920d87fd88" containerName="manager" Jul 21 06:37:33.193400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.193315 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="79f0cd21-db64-49a3-8687-fe920d87fd88" containerName="manager" Jul 21 06:37:33.193400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.193374 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d41f0c3d-6fd5-454b-9f40-9c4abeb088fb" containerName="cleanup" Jul 21 06:37:33.193400 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.193382 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="79f0cd21-db64-49a3-8687-fe920d87fd88" containerName="manager" Jul 21 06:37:33.196468 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.196452 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx" Jul 21 06:37:33.199559 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.199531 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 06:37:33.199700 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.199565 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 06:37:33.199700 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.199577 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dsmhd\"" Jul 21 06:37:33.206665 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.206641 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx"] Jul 21 06:37:33.285207 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.285164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66391c1a-0c11-4fef-b90a-715f00d51683-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kvnrx\" (UID: \"66391c1a-0c11-4fef-b90a-715f00d51683\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx" Jul 21 06:37:33.285407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.285324 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft6sv\" (UniqueName: \"kubernetes.io/projected/66391c1a-0c11-4fef-b90a-715f00d51683-kube-api-access-ft6sv\") pod \"kuadrant-operator-controller-manager-86878c769f-kvnrx\" (UID: \"66391c1a-0c11-4fef-b90a-715f00d51683\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx" Jul 21 06:37:33.385797 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.385754 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66391c1a-0c11-4fef-b90a-715f00d51683-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kvnrx\" (UID: \"66391c1a-0c11-4fef-b90a-715f00d51683\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx" Jul 21 06:37:33.385975 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.385858 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft6sv\" (UniqueName: \"kubernetes.io/projected/66391c1a-0c11-4fef-b90a-715f00d51683-kube-api-access-ft6sv\") pod \"kuadrant-operator-controller-manager-86878c769f-kvnrx\" (UID: \"66391c1a-0c11-4fef-b90a-715f00d51683\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx" Jul 21 06:37:33.386252 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.386200 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/66391c1a-0c11-4fef-b90a-715f00d51683-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kvnrx\" (UID: \"66391c1a-0c11-4fef-b90a-715f00d51683\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx" Jul 21 06:37:33.396881 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.396849 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft6sv\" (UniqueName: \"kubernetes.io/projected/66391c1a-0c11-4fef-b90a-715f00d51683-kube-api-access-ft6sv\") pod \"kuadrant-operator-controller-manager-86878c769f-kvnrx\" (UID: \"66391c1a-0c11-4fef-b90a-715f00d51683\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx" Jul 21 06:37:33.507208 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.507118 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx" Jul 21 06:37:33.645510 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.645486 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx"] Jul 21 06:37:33.646841 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:37:33.646809 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66391c1a_0c11_4fef_b90a_715f00d51683.slice/crio-e6c7354204092437032c6a3940ff4908b29d9da12239065985b4637f6af4b46f WatchSource:0}: Error finding container e6c7354204092437032c6a3940ff4908b29d9da12239065985b4637f6af4b46f: Status 404 returned error can't find the container with id e6c7354204092437032c6a3940ff4908b29d9da12239065985b4637f6af4b46f Jul 21 06:37:33.940461 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.940424 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx" event={"ID":"66391c1a-0c11-4fef-b90a-715f00d51683","Type":"ContainerStarted","Data":"55f0de394cda5433a74621cf20db55b6ce20dc242087b84aad73912e3978b7fc"} Jul 21 06:37:33.940646 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.940486 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx" event={"ID":"66391c1a-0c11-4fef-b90a-715f00d51683","Type":"ContainerStarted","Data":"e6c7354204092437032c6a3940ff4908b29d9da12239065985b4637f6af4b46f"} Jul 21 06:37:33.940646 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.940563 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx" Jul 21 06:37:33.962985 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:33.962933 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx" podStartSLOduration=0.962915431 podStartE2EDuration="962.915431ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:37:33.960270102 +0000 UTC m=+1183.905908760" watchObservedRunningTime="2026-07-21 06:37:33.962915431 +0000 UTC m=+1183.908554088" Jul 21 06:37:44.948657 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:37:44.948622 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvnrx" Jul 21 06:44:11.561438 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.561405 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2"] Jul 21 06:44:11.565181 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.565160 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.567965 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.567946 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-93c1870b-openshift-default-dockercfg-92mbn\"" Jul 21 06:44:11.585304 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.585277 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2"] Jul 21 06:44:11.704517 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.704485 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d5g2\" (UniqueName: \"kubernetes.io/projected/36fa2d93-7af1-4975-afaf-ea14b6068496-kube-api-access-8d5g2\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.704683 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.704526 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-workload-socket\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.704683 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.704600 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-token\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.704683 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.704661 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-data\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.704793 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.704723 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-credential-socket\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.704793 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.704774 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-envoy\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.704793 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.704791 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36fa2d93-7af1-4975-afaf-ea14b6068496-istiod-ca-cert\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.704888 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.704808 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-workload-certs\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.704888 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.704839 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-podinfo\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.805794 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.805757 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-workload-socket\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.805967 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.805810 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-token\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.805967 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.805868 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-data\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.805967 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.805900 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-credential-socket\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.805967 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.805942 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-envoy\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.805967 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.805959 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36fa2d93-7af1-4975-afaf-ea14b6068496-istiod-ca-cert\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.806254 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.805976 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-workload-certs\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.806254 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.806001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-podinfo\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.806254 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.806175 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d5g2\" (UniqueName: \"kubernetes.io/projected/36fa2d93-7af1-4975-afaf-ea14b6068496-kube-api-access-8d5g2\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.806431 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.806253 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-workload-socket\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.806486 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.806446 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-credential-socket\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.806544 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.806499 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-workload-certs\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.806544 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.806539 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-data\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.806841 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.806818 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36fa2d93-7af1-4975-afaf-ea14b6068496-istiod-ca-cert\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.808419 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.808392 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-envoy\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.808515 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.808484 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-podinfo\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.814022 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.813970 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-token\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.814137 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.814117 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d5g2\" (UniqueName: \"kubernetes.io/projected/36fa2d93-7af1-4975-afaf-ea14b6068496-kube-api-access-8d5g2\") pod \"e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:11.875545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:11.875511 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:12.022389 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:12.022362 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2"] Jul 21 06:44:12.023146 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:44:12.023118 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36fa2d93_7af1_4975_afaf_ea14b6068496.slice/crio-aacc6fc071b9de90db022db03f4110cc8a4f7955eaa4e564c4f0541dc5d4aa5e WatchSource:0}: Error finding container aacc6fc071b9de90db022db03f4110cc8a4f7955eaa4e564c4f0541dc5d4aa5e: Status 404 returned error can't find the container with id aacc6fc071b9de90db022db03f4110cc8a4f7955eaa4e564c4f0541dc5d4aa5e Jul 21 06:44:12.025265 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:12.025234 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:44:12.025356 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:12.025295 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:44:12.025356 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:12.025327 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:44:12.155636 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:12.155595 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" event={"ID":"36fa2d93-7af1-4975-afaf-ea14b6068496","Type":"ContainerStarted","Data":"54b4f9a5a061a71eadd8509dbd00d7bdbe955df5227d096686dbff9ab9d443dd"} Jul 21 06:44:12.155816 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:12.155656 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" event={"ID":"36fa2d93-7af1-4975-afaf-ea14b6068496","Type":"ContainerStarted","Data":"aacc6fc071b9de90db022db03f4110cc8a4f7955eaa4e564c4f0541dc5d4aa5e"} Jul 21 06:44:12.177368 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:12.177319 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" podStartSLOduration=1.177303514 podStartE2EDuration="1.177303514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:44:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:44:12.175050099 +0000 UTC m=+1582.120688755" watchObservedRunningTime="2026-07-21 06:44:12.177303514 +0000 UTC m=+1582.122942170" Jul 21 06:44:12.876018 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:12.875988 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:13.880890 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:13.880857 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:14.166762 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:14.166669 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:14.167674 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:14.167657 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:26.240019 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:26.239910 2563 generic.go:356] "Generic (PLEG): container finished" podID="36fa2d93-7af1-4975-afaf-ea14b6068496" containerID="54b4f9a5a061a71eadd8509dbd00d7bdbe955df5227d096686dbff9ab9d443dd" exitCode=0 Jul 21 06:44:26.240019 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:26.239975 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" event={"ID":"36fa2d93-7af1-4975-afaf-ea14b6068496","Type":"ContainerDied","Data":"54b4f9a5a061a71eadd8509dbd00d7bdbe955df5227d096686dbff9ab9d443dd"} Jul 21 06:44:26.240636 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:26.240500 2563 scope.go:117] "RemoveContainer" containerID="54b4f9a5a061a71eadd8509dbd00d7bdbe955df5227d096686dbff9ab9d443dd" Jul 21 06:44:26.241688 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:26.241655 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:44:26.241785 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:26.241744 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:44:26.241825 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:26.241796 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:44:27.251996 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:27.251951 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" event={"ID":"36fa2d93-7af1-4975-afaf-ea14b6068496","Type":"ContainerStarted","Data":"f5504998cc0fbbc9cdae35961e8e26928c25a6fe70bd00158063af7996f71d1c"} Jul 21 06:44:27.875988 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:27.875951 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:27.876991 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:27.876964 2563 patch_prober.go:28] interesting pod/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 21 06:44:27.877113 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:27.877025 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" podUID="36fa2d93-7af1-4975-afaf-ea14b6068496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 21 06:44:28.876791 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:28.876752 2563 patch_prober.go:28] interesting pod/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 21 06:44:28.877189 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:28.876809 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" podUID="36fa2d93-7af1-4975-afaf-ea14b6068496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 21 06:44:29.166862 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:29.166774 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:29.875988 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:29.875950 2563 patch_prober.go:28] interesting pod/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 21 06:44:29.876183 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:29.876024 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" podUID="36fa2d93-7af1-4975-afaf-ea14b6068496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 21 06:44:30.876763 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:30.876729 2563 patch_prober.go:28] interesting pod/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 21 06:44:30.877144 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:30.876781 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" podUID="36fa2d93-7af1-4975-afaf-ea14b6068496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 21 06:44:31.879884 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:31.879852 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:31.881058 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:31.881036 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:39.305008 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:39.304972 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4"] Jul 21 06:44:39.308954 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:39.308937 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4" Jul 21 06:44:39.313087 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:39.313062 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jvhnq\"" Jul 21 06:44:39.313495 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:39.313292 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 06:44:39.315144 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:39.315120 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4"] Jul 21 06:44:39.377794 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:39.377750 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1b2f321-c042-4564-8ca6-e046837c920f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4\" (UID: \"d1b2f321-c042-4564-8ca6-e046837c920f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4" Jul 21 06:44:39.479133 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:39.479093 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1b2f321-c042-4564-8ca6-e046837c920f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4\" (UID: \"d1b2f321-c042-4564-8ca6-e046837c920f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4" Jul 21 06:44:39.479758 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:39.479736 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1b2f321-c042-4564-8ca6-e046837c920f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4\" (UID: \"d1b2f321-c042-4564-8ca6-e046837c920f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4" Jul 21 06:44:39.623002 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:39.622966 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4" Jul 21 06:44:39.759838 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:39.759812 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4"] Jul 21 06:44:39.761474 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:44:39.761448 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1b2f321_c042_4564_8ca6_e046837c920f.slice/crio-a6697a798dfb543c16761a390e1f4d073fa3858b3273d811fe4965a9cd1a2055 WatchSource:0}: Error finding container a6697a798dfb543c16761a390e1f4d073fa3858b3273d811fe4965a9cd1a2055: Status 404 returned error can't find the container with id a6697a798dfb543c16761a390e1f4d073fa3858b3273d811fe4965a9cd1a2055 Jul 21 06:44:40.327699 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:40.327666 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4" event={"ID":"d1b2f321-c042-4564-8ca6-e046837c920f","Type":"ContainerStarted","Data":"ec0327f46b57c260e37835f9fc216e0532245cb641c65f1fce2e3643ddf06b36"} Jul 21 06:44:40.328059 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:40.327718 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4" event={"ID":"d1b2f321-c042-4564-8ca6-e046837c920f","Type":"ContainerStarted","Data":"a6697a798dfb543c16761a390e1f4d073fa3858b3273d811fe4965a9cd1a2055"} Jul 21 06:44:40.346003 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:40.345954 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4" podStartSLOduration=1.345937958 podStartE2EDuration="1.345937958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:44:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:44:40.343446026 +0000 UTC m=+1610.289084683" watchObservedRunningTime="2026-07-21 06:44:40.345937958 +0000 UTC m=+1610.291576641" Jul 21 06:44:41.335105 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:41.335022 2563 generic.go:356] "Generic (PLEG): container finished" podID="d1b2f321-c042-4564-8ca6-e046837c920f" containerID="ec0327f46b57c260e37835f9fc216e0532245cb641c65f1fce2e3643ddf06b36" exitCode=0 Jul 21 06:44:41.335526 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:41.335128 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4" event={"ID":"d1b2f321-c042-4564-8ca6-e046837c920f","Type":"ContainerDied","Data":"ec0327f46b57c260e37835f9fc216e0532245cb641c65f1fce2e3643ddf06b36"} Jul 21 06:44:42.475928 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:42.475904 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4" Jul 21 06:44:42.607120 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:42.607058 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1b2f321-c042-4564-8ca6-e046837c920f-maas-api-service-ca\") pod \"d1b2f321-c042-4564-8ca6-e046837c920f\" (UID: \"d1b2f321-c042-4564-8ca6-e046837c920f\") " Jul 21 06:44:42.607418 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:42.607396 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1b2f321-c042-4564-8ca6-e046837c920f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d1b2f321-c042-4564-8ca6-e046837c920f" (UID: "d1b2f321-c042-4564-8ca6-e046837c920f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:44:42.708192 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:42.708164 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1b2f321-c042-4564-8ca6-e046837c920f-maas-api-service-ca\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:44:43.347029 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:43.346995 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4" event={"ID":"d1b2f321-c042-4564-8ca6-e046837c920f","Type":"ContainerDied","Data":"a6697a798dfb543c16761a390e1f4d073fa3858b3273d811fe4965a9cd1a2055"} Jul 21 06:44:43.347194 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:43.347036 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4" Jul 21 06:44:43.347194 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:43.347043 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6697a798dfb543c16761a390e1f4d073fa3858b3273d811fe4965a9cd1a2055" Jul 21 06:44:54.419246 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:54.419194 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4"] Jul 21 06:44:54.421784 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:54.421763 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-93c1870b-6jrp4"] Jul 21 06:44:54.487012 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:54.486982 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1b2f321-c042-4564-8ca6-e046837c920f" path="/var/lib/kubelet/pods/d1b2f321-c042-4564-8ca6-e046837c920f/volumes" Jul 21 06:44:54.594441 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:54.594411 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2"] Jul 21 06:44:54.594672 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:54.594636 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" podUID="36fa2d93-7af1-4975-afaf-ea14b6068496" containerName="istio-proxy" containerID="cri-o://f5504998cc0fbbc9cdae35961e8e26928c25a6fe70bd00158063af7996f71d1c" gracePeriod=30 Jul 21 06:44:56.041884 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.041853 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d"] Jul 21 06:44:56.042299 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.042285 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b2f321-c042-4564-8ca6-e046837c920f" containerName="revoke-keys" Jul 21 06:44:56.042349 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.042300 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b2f321-c042-4564-8ca6-e046837c920f" containerName="revoke-keys" Jul 21 06:44:56.042404 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.042395 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1b2f321-c042-4564-8ca6-e046837c920f" containerName="revoke-keys" Jul 21 06:44:56.045678 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.045661 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.048666 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.048644 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-72493889-gw-openshift-default-dockercfg-2hb7v\"" Jul 21 06:44:56.059742 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.059715 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d"] Jul 21 06:44:56.139339 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.139298 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-token\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.139538 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.139398 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mmb2\" (UniqueName: \"kubernetes.io/projected/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-kube-api-access-5mmb2\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.139538 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.139465 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-workload-socket\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.139538 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.139504 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-podinfo\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.139711 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.139551 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-workload-certs\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.139711 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.139588 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istiod-ca-cert\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.139711 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.139619 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-envoy\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.139711 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.139648 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-credential-socket\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.139711 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.139702 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-data\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.240834 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.240797 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-podinfo\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.241028 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.240854 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-workload-certs\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.241028 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.240887 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istiod-ca-cert\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.241028 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.240911 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-envoy\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.241369 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.241339 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-workload-certs\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.241525 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.241387 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-credential-socket\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.241525 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.241471 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-data\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.241525 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.241522 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-token\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.241872 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.241581 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mmb2\" (UniqueName: \"kubernetes.io/projected/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-kube-api-access-5mmb2\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.241872 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.241653 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-workload-socket\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.241872 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.241704 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-credential-socket\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.241872 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.241732 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istiod-ca-cert\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.241872 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.241812 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-data\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.242173 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.241977 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-workload-socket\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.243539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.243516 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-envoy\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.243755 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.243735 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-podinfo\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.250079 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.250058 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-token\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.250397 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.250374 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mmb2\" (UniqueName: \"kubernetes.io/projected/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-kube-api-access-5mmb2\") pod \"e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.355784 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.355754 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:56.504640 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.504619 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d"] Jul 21 06:44:56.505873 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:44:56.505851 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ddaf7c_1699_4d73_b767_5f2d9384bdcb.slice/crio-ca7116d3d8447d6cf69c079a1bec5e99ff7a8641eea06ab72d647df27901094e WatchSource:0}: Error finding container ca7116d3d8447d6cf69c079a1bec5e99ff7a8641eea06ab72d647df27901094e: Status 404 returned error can't find the container with id ca7116d3d8447d6cf69c079a1bec5e99ff7a8641eea06ab72d647df27901094e Jul 21 06:44:56.507964 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.507931 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:44:56.508058 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.507992 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:44:56.508058 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:56.508019 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:44:57.433953 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:57.433917 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" event={"ID":"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb","Type":"ContainerStarted","Data":"b5a7035cdc0fcfb1e97a52e3b16866eec3f953fadfb70c810b1c67247ad1de6d"} Jul 21 06:44:57.434466 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:57.433975 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" event={"ID":"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb","Type":"ContainerStarted","Data":"ca7116d3d8447d6cf69c079a1bec5e99ff7a8641eea06ab72d647df27901094e"} Jul 21 06:44:57.457282 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:57.457237 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" podStartSLOduration=1.457207943 podStartE2EDuration="1.457207943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:44:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:44:57.454266156 +0000 UTC m=+1627.399904812" watchObservedRunningTime="2026-07-21 06:44:57.457207943 +0000 UTC m=+1627.402846600" Jul 21 06:44:58.356714 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:58.356682 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:58.361692 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:58.361667 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:58.441539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:58.441511 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:58.447041 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:58.447012 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:44:59.860454 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.860397 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:44:59.976354 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.976321 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-credential-socket\") pod \"36fa2d93-7af1-4975-afaf-ea14b6068496\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " Jul 21 06:44:59.976519 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.976361 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8d5g2\" (UniqueName: \"kubernetes.io/projected/36fa2d93-7af1-4975-afaf-ea14b6068496-kube-api-access-8d5g2\") pod \"36fa2d93-7af1-4975-afaf-ea14b6068496\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " Jul 21 06:44:59.976519 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.976386 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-data\") pod \"36fa2d93-7af1-4975-afaf-ea14b6068496\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " Jul 21 06:44:59.976519 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.976415 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-token\") pod \"36fa2d93-7af1-4975-afaf-ea14b6068496\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " Jul 21 06:44:59.976519 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.976456 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-podinfo\") pod \"36fa2d93-7af1-4975-afaf-ea14b6068496\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " Jul 21 06:44:59.976519 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.976486 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-workload-certs\") pod \"36fa2d93-7af1-4975-afaf-ea14b6068496\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " Jul 21 06:44:59.976519 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.976508 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36fa2d93-7af1-4975-afaf-ea14b6068496-istiod-ca-cert\") pod \"36fa2d93-7af1-4975-afaf-ea14b6068496\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " Jul 21 06:44:59.976841 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.976555 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-workload-socket\") pod \"36fa2d93-7af1-4975-afaf-ea14b6068496\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " Jul 21 06:44:59.976841 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.976578 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-envoy\") pod \"36fa2d93-7af1-4975-afaf-ea14b6068496\" (UID: \"36fa2d93-7af1-4975-afaf-ea14b6068496\") " Jul 21 06:44:59.976841 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.976607 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "36fa2d93-7af1-4975-afaf-ea14b6068496" (UID: "36fa2d93-7af1-4975-afaf-ea14b6068496"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:44:59.976841 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.976770 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "36fa2d93-7af1-4975-afaf-ea14b6068496" (UID: "36fa2d93-7af1-4975-afaf-ea14b6068496"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:44:59.977049 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.976881 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "36fa2d93-7af1-4975-afaf-ea14b6068496" (UID: "36fa2d93-7af1-4975-afaf-ea14b6068496"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:44:59.977049 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.976987 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-workload-certs\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:44:59.977049 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.977011 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-workload-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:44:59.977049 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.977029 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-credential-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:44:59.977270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.977249 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36fa2d93-7af1-4975-afaf-ea14b6068496-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "36fa2d93-7af1-4975-afaf-ea14b6068496" (UID: "36fa2d93-7af1-4975-afaf-ea14b6068496"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:44:59.977436 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.977412 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-data" (OuterVolumeSpecName: "istio-data") pod "36fa2d93-7af1-4975-afaf-ea14b6068496" (UID: "36fa2d93-7af1-4975-afaf-ea14b6068496"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:44:59.978849 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.978824 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "36fa2d93-7af1-4975-afaf-ea14b6068496" (UID: "36fa2d93-7af1-4975-afaf-ea14b6068496"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 06:44:59.978972 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.978934 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "36fa2d93-7af1-4975-afaf-ea14b6068496" (UID: "36fa2d93-7af1-4975-afaf-ea14b6068496"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:44:59.979257 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.979232 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-token" (OuterVolumeSpecName: "istio-token") pod "36fa2d93-7af1-4975-afaf-ea14b6068496" (UID: "36fa2d93-7af1-4975-afaf-ea14b6068496"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:44:59.979257 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:44:59.979254 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36fa2d93-7af1-4975-afaf-ea14b6068496-kube-api-access-8d5g2" (OuterVolumeSpecName: "kube-api-access-8d5g2") pod "36fa2d93-7af1-4975-afaf-ea14b6068496" (UID: "36fa2d93-7af1-4975-afaf-ea14b6068496"). InnerVolumeSpecName "kube-api-access-8d5g2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:45:00.078436 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.078406 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-token\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:00.078436 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.078431 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-podinfo\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:00.078436 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.078443 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36fa2d93-7af1-4975-afaf-ea14b6068496-istiod-ca-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:00.078662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.078452 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-envoy\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:00.078662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.078460 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d5g2\" (UniqueName: \"kubernetes.io/projected/36fa2d93-7af1-4975-afaf-ea14b6068496-kube-api-access-8d5g2\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:00.078662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.078468 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36fa2d93-7af1-4975-afaf-ea14b6068496-istio-data\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:00.457509 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.457477 2563 generic.go:356] "Generic (PLEG): container finished" podID="36fa2d93-7af1-4975-afaf-ea14b6068496" containerID="f5504998cc0fbbc9cdae35961e8e26928c25a6fe70bd00158063af7996f71d1c" exitCode=0 Jul 21 06:45:00.457681 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.457526 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" Jul 21 06:45:00.457681 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.457566 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" event={"ID":"36fa2d93-7af1-4975-afaf-ea14b6068496","Type":"ContainerDied","Data":"f5504998cc0fbbc9cdae35961e8e26928c25a6fe70bd00158063af7996f71d1c"} Jul 21 06:45:00.457681 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.457627 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2" event={"ID":"36fa2d93-7af1-4975-afaf-ea14b6068496","Type":"ContainerDied","Data":"aacc6fc071b9de90db022db03f4110cc8a4f7955eaa4e564c4f0541dc5d4aa5e"} Jul 21 06:45:00.457681 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.457657 2563 scope.go:117] "RemoveContainer" containerID="f5504998cc0fbbc9cdae35961e8e26928c25a6fe70bd00158063af7996f71d1c" Jul 21 06:45:00.469883 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.469851 2563 scope.go:117] "RemoveContainer" containerID="54b4f9a5a061a71eadd8509dbd00d7bdbe955df5227d096686dbff9ab9d443dd" Jul 21 06:45:00.478905 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.478886 2563 scope.go:117] "RemoveContainer" containerID="f5504998cc0fbbc9cdae35961e8e26928c25a6fe70bd00158063af7996f71d1c" Jul 21 06:45:00.479190 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:45:00.479168 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5504998cc0fbbc9cdae35961e8e26928c25a6fe70bd00158063af7996f71d1c\": container with ID starting with f5504998cc0fbbc9cdae35961e8e26928c25a6fe70bd00158063af7996f71d1c not found: ID does not exist" containerID="f5504998cc0fbbc9cdae35961e8e26928c25a6fe70bd00158063af7996f71d1c" Jul 21 06:45:00.479261 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.479199 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5504998cc0fbbc9cdae35961e8e26928c25a6fe70bd00158063af7996f71d1c"} err="failed to get container status \"f5504998cc0fbbc9cdae35961e8e26928c25a6fe70bd00158063af7996f71d1c\": rpc error: code = NotFound desc = could not find container \"f5504998cc0fbbc9cdae35961e8e26928c25a6fe70bd00158063af7996f71d1c\": container with ID starting with f5504998cc0fbbc9cdae35961e8e26928c25a6fe70bd00158063af7996f71d1c not found: ID does not exist" Jul 21 06:45:00.479261 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.479234 2563 scope.go:117] "RemoveContainer" containerID="54b4f9a5a061a71eadd8509dbd00d7bdbe955df5227d096686dbff9ab9d443dd" Jul 21 06:45:00.479481 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:45:00.479462 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54b4f9a5a061a71eadd8509dbd00d7bdbe955df5227d096686dbff9ab9d443dd\": container with ID starting with 54b4f9a5a061a71eadd8509dbd00d7bdbe955df5227d096686dbff9ab9d443dd not found: ID does not exist" containerID="54b4f9a5a061a71eadd8509dbd00d7bdbe955df5227d096686dbff9ab9d443dd" Jul 21 06:45:00.479534 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.479486 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54b4f9a5a061a71eadd8509dbd00d7bdbe955df5227d096686dbff9ab9d443dd"} err="failed to get container status \"54b4f9a5a061a71eadd8509dbd00d7bdbe955df5227d096686dbff9ab9d443dd\": rpc error: code = NotFound desc = could not find container \"54b4f9a5a061a71eadd8509dbd00d7bdbe955df5227d096686dbff9ab9d443dd\": container with ID starting with 54b4f9a5a061a71eadd8509dbd00d7bdbe955df5227d096686dbff9ab9d443dd not found: ID does not exist" Jul 21 06:45:00.488278 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.488259 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2"] Jul 21 06:45:00.488368 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:00.488285 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-93c1870b-openshift-default-7d5589fddf-jxtg2"] Jul 21 06:45:02.486534 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:02.486505 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36fa2d93-7af1-4975-afaf-ea14b6068496" path="/var/lib/kubelet/pods/36fa2d93-7af1-4975-afaf-ea14b6068496/volumes" Jul 21 06:45:10.519421 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:10.519382 2563 generic.go:356] "Generic (PLEG): container finished" podID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" containerID="b5a7035cdc0fcfb1e97a52e3b16866eec3f953fadfb70c810b1c67247ad1de6d" exitCode=0 Jul 21 06:45:10.519829 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:10.519458 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" event={"ID":"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb","Type":"ContainerDied","Data":"b5a7035cdc0fcfb1e97a52e3b16866eec3f953fadfb70c810b1c67247ad1de6d"} Jul 21 06:45:10.519874 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:10.519844 2563 scope.go:117] "RemoveContainer" containerID="b5a7035cdc0fcfb1e97a52e3b16866eec3f953fadfb70c810b1c67247ad1de6d" Jul 21 06:45:10.520869 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:10.520838 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:45:10.520935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:10.520919 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:45:10.520970 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:10.520950 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:45:11.527617 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:11.527584 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" event={"ID":"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb","Type":"ContainerStarted","Data":"f4d7d51389d522153dcc9dd4414af7014a6924b4d110a3109ca45862552947dd"} Jul 21 06:45:12.357419 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:12.357381 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:45:12.357827 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:12.357643 2563 patch_prober.go:28] interesting pod/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 21 06:45:12.357827 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:12.357713 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" podUID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 21 06:45:13.357660 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:13.357186 2563 patch_prober.go:28] interesting pod/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 21 06:45:13.357660 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:13.357274 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" podUID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 21 06:45:13.441325 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:13.441293 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:45:14.357117 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:14.357078 2563 patch_prober.go:28] interesting pod/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 21 06:45:14.357452 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:14.357412 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" podUID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 21 06:45:15.356605 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:15.356566 2563 patch_prober.go:28] interesting pod/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 21 06:45:15.357205 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:15.357176 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" podUID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 21 06:45:16.360342 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:16.360308 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:45:16.361440 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:16.361416 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:45:24.046850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.046819 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s"] Jul 21 06:45:24.047239 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.047211 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36fa2d93-7af1-4975-afaf-ea14b6068496" containerName="istio-proxy" Jul 21 06:45:24.047239 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.047240 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="36fa2d93-7af1-4975-afaf-ea14b6068496" containerName="istio-proxy" Jul 21 06:45:24.047345 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.047336 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="36fa2d93-7af1-4975-afaf-ea14b6068496" containerName="istio-proxy" Jul 21 06:45:24.047387 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.047351 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="36fa2d93-7af1-4975-afaf-ea14b6068496" containerName="istio-proxy" Jul 21 06:45:24.050353 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.050337 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s" Jul 21 06:45:24.054675 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.054657 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 06:45:24.054789 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.054698 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jvhnq\"" Jul 21 06:45:24.063753 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.063728 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s"] Jul 21 06:45:24.116804 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.116765 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11fa3a68-f40d-448c-8bfa-6484887a1b1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-72493889-9582s\" (UID: \"11fa3a68-f40d-448c-8bfa-6484887a1b1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s" Jul 21 06:45:24.217978 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.217944 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11fa3a68-f40d-448c-8bfa-6484887a1b1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-72493889-9582s\" (UID: \"11fa3a68-f40d-448c-8bfa-6484887a1b1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s" Jul 21 06:45:24.218575 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.218557 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11fa3a68-f40d-448c-8bfa-6484887a1b1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-72493889-9582s\" (UID: \"11fa3a68-f40d-448c-8bfa-6484887a1b1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s" Jul 21 06:45:24.359933 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.359896 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s" Jul 21 06:45:24.500662 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.500637 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s"] Jul 21 06:45:24.501836 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:45:24.501808 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11fa3a68_f40d_448c_8bfa_6484887a1b1b.slice/crio-51800cbb3a68ba2ed96f83a7f08e57ef20e923eff6b92a8dd839295d397291ef WatchSource:0}: Error finding container 51800cbb3a68ba2ed96f83a7f08e57ef20e923eff6b92a8dd839295d397291ef: Status 404 returned error can't find the container with id 51800cbb3a68ba2ed96f83a7f08e57ef20e923eff6b92a8dd839295d397291ef Jul 21 06:45:24.605964 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:24.605935 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s" event={"ID":"11fa3a68-f40d-448c-8bfa-6484887a1b1b","Type":"ContainerStarted","Data":"51800cbb3a68ba2ed96f83a7f08e57ef20e923eff6b92a8dd839295d397291ef"} Jul 21 06:45:25.614112 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:25.614072 2563 generic.go:356] "Generic (PLEG): container finished" podID="11fa3a68-f40d-448c-8bfa-6484887a1b1b" containerID="e66ab4f1162fef9f2d0c5051de8f9a08ebb9e144038cfbca19758b9a298f830c" exitCode=0 Jul 21 06:45:25.614567 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:25.614162 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s" event={"ID":"11fa3a68-f40d-448c-8bfa-6484887a1b1b","Type":"ContainerDied","Data":"e66ab4f1162fef9f2d0c5051de8f9a08ebb9e144038cfbca19758b9a298f830c"} Jul 21 06:45:26.750896 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:26.750868 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s" Jul 21 06:45:26.845161 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:26.845130 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11fa3a68-f40d-448c-8bfa-6484887a1b1b-maas-api-service-ca\") pod \"11fa3a68-f40d-448c-8bfa-6484887a1b1b\" (UID: \"11fa3a68-f40d-448c-8bfa-6484887a1b1b\") " Jul 21 06:45:26.845485 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:26.845457 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11fa3a68-f40d-448c-8bfa-6484887a1b1b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "11fa3a68-f40d-448c-8bfa-6484887a1b1b" (UID: "11fa3a68-f40d-448c-8bfa-6484887a1b1b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:45:26.845597 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:26.845571 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11fa3a68-f40d-448c-8bfa-6484887a1b1b-maas-api-service-ca\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:27.626294 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:27.626263 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s" Jul 21 06:45:27.626294 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:27.626285 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s" event={"ID":"11fa3a68-f40d-448c-8bfa-6484887a1b1b","Type":"ContainerDied","Data":"51800cbb3a68ba2ed96f83a7f08e57ef20e923eff6b92a8dd839295d397291ef"} Jul 21 06:45:27.626511 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:27.626337 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51800cbb3a68ba2ed96f83a7f08e57ef20e923eff6b92a8dd839295d397291ef" Jul 21 06:45:39.120551 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:39.120516 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s"] Jul 21 06:45:39.125232 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:39.125179 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-72493889-9582s"] Jul 21 06:45:39.286426 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:39.286396 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d"] Jul 21 06:45:39.286642 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:39.286617 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" podUID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" containerName="istio-proxy" containerID="cri-o://f4d7d51389d522153dcc9dd4414af7014a6924b4d110a3109ca45862552947dd" gracePeriod=30 Jul 21 06:45:40.133468 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.133433 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf"] Jul 21 06:45:40.133873 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.133856 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11fa3a68-f40d-448c-8bfa-6484887a1b1b" containerName="revoke-keys" Jul 21 06:45:40.133873 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.133868 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="11fa3a68-f40d-448c-8bfa-6484887a1b1b" containerName="revoke-keys" Jul 21 06:45:40.133961 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.133887 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36fa2d93-7af1-4975-afaf-ea14b6068496" containerName="istio-proxy" Jul 21 06:45:40.133961 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.133894 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="36fa2d93-7af1-4975-afaf-ea14b6068496" containerName="istio-proxy" Jul 21 06:45:40.134038 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.133963 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="11fa3a68-f40d-448c-8bfa-6484887a1b1b" containerName="revoke-keys" Jul 21 06:45:40.137511 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.137491 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.141157 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.141136 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-281e53fe-openshift-default-dockercfg-8pgqg\"" Jul 21 06:45:40.165581 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.165556 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf"] Jul 21 06:45:40.275519 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.275484 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghwp7\" (UniqueName: \"kubernetes.io/projected/8c7869ea-3cb9-4601-8b6a-249367a97086-kube-api-access-ghwp7\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.275688 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.275539 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-workload-socket\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.275688 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.275634 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-credential-socket\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.275688 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.275652 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-workload-certs\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.275688 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.275681 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7869ea-3cb9-4601-8b6a-249367a97086-istiod-ca-cert\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.275854 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.275741 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-data\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.275854 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.275781 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-token\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.275854 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.275817 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-envoy\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.275854 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.275841 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-podinfo\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.377013 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.376979 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-podinfo\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.377200 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.377022 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghwp7\" (UniqueName: \"kubernetes.io/projected/8c7869ea-3cb9-4601-8b6a-249367a97086-kube-api-access-ghwp7\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.377200 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.377150 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-workload-socket\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.377373 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.377266 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-credential-socket\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.377373 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.377295 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-workload-certs\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.377373 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.377347 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7869ea-3cb9-4601-8b6a-249367a97086-istiod-ca-cert\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.377592 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.377493 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-data\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.377592 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.377549 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-workload-socket\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.377592 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.377558 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-token\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.377753 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.377614 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-credential-socket\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.377753 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.377623 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-envoy\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.377753 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.377691 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-workload-certs\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.377932 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.377843 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-data\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.378065 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.378045 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7869ea-3cb9-4601-8b6a-249367a97086-istiod-ca-cert\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.379657 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.379637 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-podinfo\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.379833 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.379814 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-envoy\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.404572 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.404488 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-token\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.407547 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.407522 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghwp7\" (UniqueName: \"kubernetes.io/projected/8c7869ea-3cb9-4601-8b6a-249367a97086-kube-api-access-ghwp7\") pod \"e2e-ait-281e53fe-openshift-default-698897c657-f75mf\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.449209 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.449183 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:40.486777 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.486748 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11fa3a68-f40d-448c-8bfa-6484887a1b1b" path="/var/lib/kubelet/pods/11fa3a68-f40d-448c-8bfa-6484887a1b1b/volumes" Jul 21 06:45:40.596464 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.596424 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf"] Jul 21 06:45:40.597440 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:45:40.597412 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c7869ea_3cb9_4601_8b6a_249367a97086.slice/crio-e40d353a430434432367f415c0c78c291c2b078315fe26c37561c67143d83736 WatchSource:0}: Error finding container e40d353a430434432367f415c0c78c291c2b078315fe26c37561c67143d83736: Status 404 returned error can't find the container with id e40d353a430434432367f415c0c78c291c2b078315fe26c37561c67143d83736 Jul 21 06:45:40.599676 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.599646 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:45:40.599756 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.599709 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:45:40.599756 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.599739 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:45:40.698513 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.698478 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" event={"ID":"8c7869ea-3cb9-4601-8b6a-249367a97086","Type":"ContainerStarted","Data":"276126d1813bac0780f4bf89e18ddaaeeaa454082ca65f445265b5eca957f138"} Jul 21 06:45:40.698664 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.698546 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" event={"ID":"8c7869ea-3cb9-4601-8b6a-249367a97086","Type":"ContainerStarted","Data":"e40d353a430434432367f415c0c78c291c2b078315fe26c37561c67143d83736"} Jul 21 06:45:40.723370 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:40.723310 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" podStartSLOduration=0.723286911 podStartE2EDuration="723.286911ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:45:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:45:40.718668677 +0000 UTC m=+1670.664307335" watchObservedRunningTime="2026-07-21 06:45:40.723286911 +0000 UTC m=+1670.668925568" Jul 21 06:45:41.450240 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:41.450185 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:41.454930 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:41.454906 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:41.704462 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:41.704373 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:41.705117 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:41.705099 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:44.548504 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.548446 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:45:44.720389 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.720356 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-envoy\") pod \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " Jul 21 06:45:44.720564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.720401 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-token\") pod \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " Jul 21 06:45:44.720564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.720457 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-workload-certs\") pod \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " Jul 21 06:45:44.720564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.720530 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-credential-socket\") pod \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " Jul 21 06:45:44.720564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.720558 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-workload-socket\") pod \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " Jul 21 06:45:44.720771 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.720583 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-podinfo\") pod \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " Jul 21 06:45:44.720771 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.720613 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-data\") pod \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " Jul 21 06:45:44.720771 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.720683 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istiod-ca-cert\") pod \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " Jul 21 06:45:44.720771 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.720735 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mmb2\" (UniqueName: \"kubernetes.io/projected/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-kube-api-access-5mmb2\") pod \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\" (UID: \"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb\") " Jul 21 06:45:44.720971 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.720842 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" (UID: "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:45:44.720971 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.720856 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" (UID: "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:45:44.721166 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.721133 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-workload-certs\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:44.721166 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.721156 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-credential-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:44.721320 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.721282 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" (UID: "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:45:44.721433 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.721379 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" (UID: "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:45:44.722140 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.722108 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" (UID: "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:45:44.722918 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.722888 2563 generic.go:356] "Generic (PLEG): container finished" podID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" containerID="f4d7d51389d522153dcc9dd4414af7014a6924b4d110a3109ca45862552947dd" exitCode=0 Jul 21 06:45:44.723050 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.722964 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" event={"ID":"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb","Type":"ContainerDied","Data":"f4d7d51389d522153dcc9dd4414af7014a6924b4d110a3109ca45862552947dd"} Jul 21 06:45:44.723050 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.722972 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" Jul 21 06:45:44.723050 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.723020 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d" event={"ID":"d7ddaf7c-1699-4d73-b767-5f2d9384bdcb","Type":"ContainerDied","Data":"ca7116d3d8447d6cf69c079a1bec5e99ff7a8641eea06ab72d647df27901094e"} Jul 21 06:45:44.723377 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.723052 2563 scope.go:117] "RemoveContainer" containerID="f4d7d51389d522153dcc9dd4414af7014a6924b4d110a3109ca45862552947dd" Jul 21 06:45:44.723377 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.723292 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" (UID: "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:45:44.723506 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.723481 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" (UID: "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:45:44.723506 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.723490 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-kube-api-access-5mmb2" (OuterVolumeSpecName: "kube-api-access-5mmb2") pod "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" (UID: "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb"). InnerVolumeSpecName "kube-api-access-5mmb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:45:44.723657 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.723638 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" (UID: "d7ddaf7c-1699-4d73-b767-5f2d9384bdcb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 06:45:44.743637 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.743617 2563 scope.go:117] "RemoveContainer" containerID="b5a7035cdc0fcfb1e97a52e3b16866eec3f953fadfb70c810b1c67247ad1de6d" Jul 21 06:45:44.752513 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.752492 2563 scope.go:117] "RemoveContainer" containerID="f4d7d51389d522153dcc9dd4414af7014a6924b4d110a3109ca45862552947dd" Jul 21 06:45:44.752813 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:45:44.752785 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4d7d51389d522153dcc9dd4414af7014a6924b4d110a3109ca45862552947dd\": container with ID starting with f4d7d51389d522153dcc9dd4414af7014a6924b4d110a3109ca45862552947dd not found: ID does not exist" containerID="f4d7d51389d522153dcc9dd4414af7014a6924b4d110a3109ca45862552947dd" Jul 21 06:45:44.752912 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.752824 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4d7d51389d522153dcc9dd4414af7014a6924b4d110a3109ca45862552947dd"} err="failed to get container status \"f4d7d51389d522153dcc9dd4414af7014a6924b4d110a3109ca45862552947dd\": rpc error: code = NotFound desc = could not find container \"f4d7d51389d522153dcc9dd4414af7014a6924b4d110a3109ca45862552947dd\": container with ID starting with f4d7d51389d522153dcc9dd4414af7014a6924b4d110a3109ca45862552947dd not found: ID does not exist" Jul 21 06:45:44.752912 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.752843 2563 scope.go:117] "RemoveContainer" containerID="b5a7035cdc0fcfb1e97a52e3b16866eec3f953fadfb70c810b1c67247ad1de6d" Jul 21 06:45:44.753103 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:45:44.753087 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5a7035cdc0fcfb1e97a52e3b16866eec3f953fadfb70c810b1c67247ad1de6d\": container with ID starting with b5a7035cdc0fcfb1e97a52e3b16866eec3f953fadfb70c810b1c67247ad1de6d not found: ID does not exist" containerID="b5a7035cdc0fcfb1e97a52e3b16866eec3f953fadfb70c810b1c67247ad1de6d" Jul 21 06:45:44.753154 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.753109 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5a7035cdc0fcfb1e97a52e3b16866eec3f953fadfb70c810b1c67247ad1de6d"} err="failed to get container status \"b5a7035cdc0fcfb1e97a52e3b16866eec3f953fadfb70c810b1c67247ad1de6d\": rpc error: code = NotFound desc = could not find container \"b5a7035cdc0fcfb1e97a52e3b16866eec3f953fadfb70c810b1c67247ad1de6d\": container with ID starting with b5a7035cdc0fcfb1e97a52e3b16866eec3f953fadfb70c810b1c67247ad1de6d not found: ID does not exist" Jul 21 06:45:44.822124 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.822093 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-workload-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:44.822124 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.822118 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-podinfo\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:44.822124 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.822128 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-data\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:44.822370 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.822137 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istiod-ca-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:44.822370 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.822145 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mmb2\" (UniqueName: \"kubernetes.io/projected/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-kube-api-access-5mmb2\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:44.822370 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.822153 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-envoy\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:44.822370 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:44.822162 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb-istio-token\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:45:45.048169 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:45.048137 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d"] Jul 21 06:45:45.056173 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:45.056148 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-72493889-gw-openshift-default-5fb486495-h6g5d"] Jul 21 06:45:46.486782 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:46.486742 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" path="/var/lib/kubelet/pods/d7ddaf7c-1699-4d73-b767-5f2d9384bdcb/volumes" Jul 21 06:45:56.706504 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:56.706473 2563 patch_prober.go:28] interesting pod/e2e-ait-281e53fe-openshift-default-698897c657-f75mf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 21 06:45:56.706888 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:56.706527 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" podUID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 21 06:45:56.801284 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:56.801248 2563 generic.go:356] "Generic (PLEG): container finished" podID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerID="276126d1813bac0780f4bf89e18ddaaeeaa454082ca65f445265b5eca957f138" exitCode=0 Jul 21 06:45:56.801460 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:56.801333 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" event={"ID":"8c7869ea-3cb9-4601-8b6a-249367a97086","Type":"ContainerDied","Data":"276126d1813bac0780f4bf89e18ddaaeeaa454082ca65f445265b5eca957f138"} Jul 21 06:45:56.801762 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:56.801748 2563 scope.go:117] "RemoveContainer" containerID="276126d1813bac0780f4bf89e18ddaaeeaa454082ca65f445265b5eca957f138" Jul 21 06:45:56.802829 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:56.802806 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:45:56.802884 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:56.802871 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:45:56.802918 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:56.802900 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:45:57.808821 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:57.808787 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" event={"ID":"8c7869ea-3cb9-4601-8b6a-249367a97086","Type":"ContainerStarted","Data":"4778c7849aabccf43b4e66bc6dfdd85a609b32746d0e756ac557d918046d221c"} Jul 21 06:45:58.450328 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:58.450289 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:45:58.450537 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:58.450421 2563 patch_prober.go:28] interesting pod/e2e-ait-281e53fe-openshift-default-698897c657-f75mf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 21 06:45:58.450537 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:58.450467 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" podUID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 21 06:45:59.449996 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:59.449952 2563 patch_prober.go:28] interesting pod/e2e-ait-281e53fe-openshift-default-698897c657-f75mf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 21 06:45:59.451098 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:45:59.450665 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" podUID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 21 06:46:00.450440 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:00.450399 2563 patch_prober.go:28] interesting pod/e2e-ait-281e53fe-openshift-default-698897c657-f75mf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 21 06:46:00.450881 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:00.450463 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" podUID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 21 06:46:01.450611 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:01.450567 2563 patch_prober.go:28] interesting pod/e2e-ait-281e53fe-openshift-default-698897c657-f75mf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 21 06:46:01.451114 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:01.450657 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" podUID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 21 06:46:02.453782 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:02.453752 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:46:02.454269 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:02.454114 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:46:02.454714 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:02.454695 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:46:24.298444 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:24.298417 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf"] Jul 21 06:46:24.298936 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:24.298653 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" podUID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerName="istio-proxy" containerID="cri-o://4778c7849aabccf43b4e66bc6dfdd85a609b32746d0e756ac557d918046d221c" gracePeriod=30 Jul 21 06:46:25.073534 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.073496 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz"] Jul 21 06:46:25.073909 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.073896 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" containerName="istio-proxy" Jul 21 06:46:25.073958 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.073912 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" containerName="istio-proxy" Jul 21 06:46:25.074009 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.074001 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" containerName="istio-proxy" Jul 21 06:46:25.074043 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.074015 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" containerName="istio-proxy" Jul 21 06:46:25.077192 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.077172 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.080474 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.080455 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-584a4a02-openshift-default-dockercfg-t97rx\"" Jul 21 06:46:25.093928 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.093905 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz"] Jul 21 06:46:25.212020 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.211983 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e883da8-628e-40c2-9345-754fbd347356-istio-token\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.212020 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.212023 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-istio-envoy\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.212340 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.212047 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e883da8-628e-40c2-9345-754fbd347356-istiod-ca-cert\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.212340 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.212174 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-workload-socket\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.212340 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.212251 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e883da8-628e-40c2-9345-754fbd347356-istio-podinfo\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.212340 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.212288 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-credential-socket\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.212340 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.212322 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2pkv\" (UniqueName: \"kubernetes.io/projected/2e883da8-628e-40c2-9345-754fbd347356-kube-api-access-d2pkv\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.212556 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.212358 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-workload-certs\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.212556 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.212419 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-istio-data\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.313731 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.313697 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-workload-socket\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.314203 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.313966 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e883da8-628e-40c2-9345-754fbd347356-istio-podinfo\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.314203 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.314008 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-credential-socket\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.314203 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.314046 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2pkv\" (UniqueName: \"kubernetes.io/projected/2e883da8-628e-40c2-9345-754fbd347356-kube-api-access-d2pkv\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.314203 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.314080 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-workload-certs\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.314203 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.314132 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-istio-data\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.314203 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.314163 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-workload-socket\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.314564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.314459 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-istio-data\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.314564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.314476 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-workload-certs\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.314564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.314506 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e883da8-628e-40c2-9345-754fbd347356-istio-token\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.314564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.314545 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-istio-envoy\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.314564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.314553 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-credential-socket\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.314810 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.314585 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e883da8-628e-40c2-9345-754fbd347356-istiod-ca-cert\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.315571 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.315535 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e883da8-628e-40c2-9345-754fbd347356-istiod-ca-cert\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.317386 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.317364 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-istio-envoy\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.317576 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.317550 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e883da8-628e-40c2-9345-754fbd347356-istio-podinfo\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.322888 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.322868 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e883da8-628e-40c2-9345-754fbd347356-istio-token\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.323013 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.322996 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2pkv\" (UniqueName: \"kubernetes.io/projected/2e883da8-628e-40c2-9345-754fbd347356-kube-api-access-d2pkv\") pod \"e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.387636 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.387581 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:25.528091 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.528067 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz"] Jul 21 06:46:25.529773 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:46:25.529753 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e883da8_628e_40c2_9345_754fbd347356.slice/crio-a27b0316de1d3e9bbcb5d89d01ebd5f30c28e277359c1ca433a4a8717fd6eb60 WatchSource:0}: Error finding container a27b0316de1d3e9bbcb5d89d01ebd5f30c28e277359c1ca433a4a8717fd6eb60: Status 404 returned error can't find the container with id a27b0316de1d3e9bbcb5d89d01ebd5f30c28e277359c1ca433a4a8717fd6eb60 Jul 21 06:46:25.532343 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.532313 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:46:25.532408 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.532376 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:46:25.532408 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.532401 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:46:25.966908 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.966878 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" event={"ID":"2e883da8-628e-40c2-9345-754fbd347356","Type":"ContainerStarted","Data":"a6e0c1fab36897391f7386f727d0faeedb3468585cb699f633b308efa1be24a8"} Jul 21 06:46:25.967074 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.966952 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" event={"ID":"2e883da8-628e-40c2-9345-754fbd347356","Type":"ContainerStarted","Data":"a27b0316de1d3e9bbcb5d89d01ebd5f30c28e277359c1ca433a4a8717fd6eb60"} Jul 21 06:46:25.987111 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:25.987062 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" podStartSLOduration=0.987048622 podStartE2EDuration="987.048622ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:46:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:46:25.985393981 +0000 UTC m=+1715.931032640" watchObservedRunningTime="2026-07-21 06:46:25.987048622 +0000 UTC m=+1715.932687278" Jul 21 06:46:26.388260 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:26.388203 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:26.393084 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:26.393063 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:26.973282 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:26.973251 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:26.974088 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:26.974067 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:29.552090 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.552065 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:46:29.554340 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.554323 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-workload-certs\") pod \"8c7869ea-3cb9-4601-8b6a-249367a97086\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " Jul 21 06:46:29.554407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.554369 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-workload-socket\") pod \"8c7869ea-3cb9-4601-8b6a-249367a97086\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " Jul 21 06:46:29.554407 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.554392 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7869ea-3cb9-4601-8b6a-249367a97086-istiod-ca-cert\") pod \"8c7869ea-3cb9-4601-8b6a-249367a97086\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " Jul 21 06:46:29.554523 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.554420 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-token\") pod \"8c7869ea-3cb9-4601-8b6a-249367a97086\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " Jul 21 06:46:29.554523 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.554479 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghwp7\" (UniqueName: \"kubernetes.io/projected/8c7869ea-3cb9-4601-8b6a-249367a97086-kube-api-access-ghwp7\") pod \"8c7869ea-3cb9-4601-8b6a-249367a97086\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " Jul 21 06:46:29.554523 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.554508 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-data\") pod \"8c7869ea-3cb9-4601-8b6a-249367a97086\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " Jul 21 06:46:29.554667 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.554533 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-podinfo\") pod \"8c7869ea-3cb9-4601-8b6a-249367a97086\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " Jul 21 06:46:29.554667 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.554538 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8c7869ea-3cb9-4601-8b6a-249367a97086" (UID: "8c7869ea-3cb9-4601-8b6a-249367a97086"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:46:29.554667 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.554563 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-envoy\") pod \"8c7869ea-3cb9-4601-8b6a-249367a97086\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " Jul 21 06:46:29.554667 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.554598 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-credential-socket\") pod \"8c7869ea-3cb9-4601-8b6a-249367a97086\" (UID: \"8c7869ea-3cb9-4601-8b6a-249367a97086\") " Jul 21 06:46:29.554856 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.554708 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8c7869ea-3cb9-4601-8b6a-249367a97086" (UID: "8c7869ea-3cb9-4601-8b6a-249367a97086"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:46:29.554856 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.554820 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c7869ea-3cb9-4601-8b6a-249367a97086-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8c7869ea-3cb9-4601-8b6a-249367a97086" (UID: "8c7869ea-3cb9-4601-8b6a-249367a97086"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:46:29.554946 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.554895 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8c7869ea-3cb9-4601-8b6a-249367a97086" (UID: "8c7869ea-3cb9-4601-8b6a-249367a97086"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:46:29.555050 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.555018 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-workload-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:46:29.555050 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.555038 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7869ea-3cb9-4601-8b6a-249367a97086-istiod-ca-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:46:29.555140 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.555053 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-credential-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:46:29.555140 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.555065 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-workload-certs\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:46:29.555457 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.555424 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-data" (OuterVolumeSpecName: "istio-data") pod "8c7869ea-3cb9-4601-8b6a-249367a97086" (UID: "8c7869ea-3cb9-4601-8b6a-249367a97086"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:46:29.557192 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.557169 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8c7869ea-3cb9-4601-8b6a-249367a97086" (UID: "8c7869ea-3cb9-4601-8b6a-249367a97086"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 06:46:29.557310 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.557196 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c7869ea-3cb9-4601-8b6a-249367a97086" (UID: "8c7869ea-3cb9-4601-8b6a-249367a97086"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:46:29.557473 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.557452 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8c7869ea-3cb9-4601-8b6a-249367a97086" (UID: "8c7869ea-3cb9-4601-8b6a-249367a97086"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:46:29.557589 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.557474 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c7869ea-3cb9-4601-8b6a-249367a97086-kube-api-access-ghwp7" (OuterVolumeSpecName: "kube-api-access-ghwp7") pod "8c7869ea-3cb9-4601-8b6a-249367a97086" (UID: "8c7869ea-3cb9-4601-8b6a-249367a97086"). InnerVolumeSpecName "kube-api-access-ghwp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:46:29.656177 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.656147 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghwp7\" (UniqueName: \"kubernetes.io/projected/8c7869ea-3cb9-4601-8b6a-249367a97086-kube-api-access-ghwp7\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:46:29.656177 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.656173 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-data\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:46:29.656177 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.656182 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-podinfo\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:46:29.656432 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.656191 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-envoy\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:46:29.656432 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.656202 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7869ea-3cb9-4601-8b6a-249367a97086-istio-token\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:46:29.993845 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.993810 2563 generic.go:356] "Generic (PLEG): container finished" podID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerID="4778c7849aabccf43b4e66bc6dfdd85a609b32746d0e756ac557d918046d221c" exitCode=0 Jul 21 06:46:29.994048 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.993864 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" Jul 21 06:46:29.994048 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.993901 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" event={"ID":"8c7869ea-3cb9-4601-8b6a-249367a97086","Type":"ContainerDied","Data":"4778c7849aabccf43b4e66bc6dfdd85a609b32746d0e756ac557d918046d221c"} Jul 21 06:46:29.994048 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.993974 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf" event={"ID":"8c7869ea-3cb9-4601-8b6a-249367a97086","Type":"ContainerDied","Data":"e40d353a430434432367f415c0c78c291c2b078315fe26c37561c67143d83736"} Jul 21 06:46:29.994048 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:29.994001 2563 scope.go:117] "RemoveContainer" containerID="4778c7849aabccf43b4e66bc6dfdd85a609b32746d0e756ac557d918046d221c" Jul 21 06:46:30.004247 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:30.004204 2563 scope.go:117] "RemoveContainer" containerID="276126d1813bac0780f4bf89e18ddaaeeaa454082ca65f445265b5eca957f138" Jul 21 06:46:30.013127 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:30.013076 2563 scope.go:117] "RemoveContainer" containerID="4778c7849aabccf43b4e66bc6dfdd85a609b32746d0e756ac557d918046d221c" Jul 21 06:46:30.013681 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:46:30.013622 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4778c7849aabccf43b4e66bc6dfdd85a609b32746d0e756ac557d918046d221c\": container with ID starting with 4778c7849aabccf43b4e66bc6dfdd85a609b32746d0e756ac557d918046d221c not found: ID does not exist" containerID="4778c7849aabccf43b4e66bc6dfdd85a609b32746d0e756ac557d918046d221c" Jul 21 06:46:30.013681 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:30.013664 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4778c7849aabccf43b4e66bc6dfdd85a609b32746d0e756ac557d918046d221c"} err="failed to get container status \"4778c7849aabccf43b4e66bc6dfdd85a609b32746d0e756ac557d918046d221c\": rpc error: code = NotFound desc = could not find container \"4778c7849aabccf43b4e66bc6dfdd85a609b32746d0e756ac557d918046d221c\": container with ID starting with 4778c7849aabccf43b4e66bc6dfdd85a609b32746d0e756ac557d918046d221c not found: ID does not exist" Jul 21 06:46:30.013852 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:30.013688 2563 scope.go:117] "RemoveContainer" containerID="276126d1813bac0780f4bf89e18ddaaeeaa454082ca65f445265b5eca957f138" Jul 21 06:46:30.014000 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:46:30.013978 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"276126d1813bac0780f4bf89e18ddaaeeaa454082ca65f445265b5eca957f138\": container with ID starting with 276126d1813bac0780f4bf89e18ddaaeeaa454082ca65f445265b5eca957f138 not found: ID does not exist" containerID="276126d1813bac0780f4bf89e18ddaaeeaa454082ca65f445265b5eca957f138" Jul 21 06:46:30.014061 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:30.014008 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276126d1813bac0780f4bf89e18ddaaeeaa454082ca65f445265b5eca957f138"} err="failed to get container status \"276126d1813bac0780f4bf89e18ddaaeeaa454082ca65f445265b5eca957f138\": rpc error: code = NotFound desc = could not find container \"276126d1813bac0780f4bf89e18ddaaeeaa454082ca65f445265b5eca957f138\": container with ID starting with 276126d1813bac0780f4bf89e18ddaaeeaa454082ca65f445265b5eca957f138 not found: ID does not exist" Jul 21 06:46:30.016453 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:30.016429 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf"] Jul 21 06:46:30.020345 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:30.020324 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-281e53fe-openshift-default-698897c657-f75mf"] Jul 21 06:46:30.487033 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:30.486984 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c7869ea-3cb9-4601-8b6a-249367a97086" path="/var/lib/kubelet/pods/8c7869ea-3cb9-4601-8b6a-249367a97086/volumes" Jul 21 06:46:41.059020 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:41.058941 2563 generic.go:356] "Generic (PLEG): container finished" podID="2e883da8-628e-40c2-9345-754fbd347356" containerID="a6e0c1fab36897391f7386f727d0faeedb3468585cb699f633b308efa1be24a8" exitCode=0 Jul 21 06:46:41.059464 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:41.059018 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" event={"ID":"2e883da8-628e-40c2-9345-754fbd347356","Type":"ContainerDied","Data":"a6e0c1fab36897391f7386f727d0faeedb3468585cb699f633b308efa1be24a8"} Jul 21 06:46:41.059464 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:41.059422 2563 scope.go:117] "RemoveContainer" containerID="a6e0c1fab36897391f7386f727d0faeedb3468585cb699f633b308efa1be24a8" Jul 21 06:46:41.060421 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:41.060393 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:46:41.060484 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:41.060464 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:46:41.060519 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:41.060498 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:46:41.973466 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:41.973437 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:42.066650 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:42.066617 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" event={"ID":"2e883da8-628e-40c2-9345-754fbd347356","Type":"ContainerStarted","Data":"b9dd537067cb19cc7f98f32296d753c4d2d6efb0d6b1b5398a2d85b411fb11b1"} Jul 21 06:46:42.388296 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:42.388262 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:42.389469 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:42.389444 2563 patch_prober.go:28] interesting pod/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 21 06:46:42.389574 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:42.389499 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" podUID="2e883da8-628e-40c2-9345-754fbd347356" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 21 06:46:43.387950 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:43.387907 2563 patch_prober.go:28] interesting pod/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 21 06:46:43.388447 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:43.387969 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" podUID="2e883da8-628e-40c2-9345-754fbd347356" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 21 06:46:44.387953 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:44.387921 2563 patch_prober.go:28] interesting pod/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 21 06:46:44.388386 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:44.387975 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" podUID="2e883da8-628e-40c2-9345-754fbd347356" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 21 06:46:45.388921 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:45.388886 2563 patch_prober.go:28] interesting pod/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 21 06:46:45.389337 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:45.388938 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" podUID="2e883da8-628e-40c2-9345-754fbd347356" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 21 06:46:46.392252 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:46.392201 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:46.392690 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:46.392542 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:46.393233 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:46.393193 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:46:52.243573 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.243540 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7"] Jul 21 06:46:52.243982 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.243966 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerName="istio-proxy" Jul 21 06:46:52.243982 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.243983 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerName="istio-proxy" Jul 21 06:46:52.244057 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.243997 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" containerName="istio-proxy" Jul 21 06:46:52.244057 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.244002 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ddaf7c-1699-4d73-b767-5f2d9384bdcb" containerName="istio-proxy" Jul 21 06:46:52.244057 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.244010 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerName="istio-proxy" Jul 21 06:46:52.244057 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.244016 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerName="istio-proxy" Jul 21 06:46:52.244182 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.244090 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerName="istio-proxy" Jul 21 06:46:52.244182 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.244098 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c7869ea-3cb9-4601-8b6a-249367a97086" containerName="istio-proxy" Jul 21 06:46:52.247125 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.247107 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7" Jul 21 06:46:52.250190 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.250147 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jvhnq\"" Jul 21 06:46:52.250360 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.250157 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 06:46:52.255474 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.255451 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7"] Jul 21 06:46:52.387019 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.386972 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e00b411-5685-41e4-8570-faa8f022c493-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7\" (UID: \"5e00b411-5685-41e4-8570-faa8f022c493\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7" Jul 21 06:46:52.488453 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.488424 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e00b411-5685-41e4-8570-faa8f022c493-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7\" (UID: \"5e00b411-5685-41e4-8570-faa8f022c493\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7" Jul 21 06:46:52.489029 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.489012 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e00b411-5685-41e4-8570-faa8f022c493-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7\" (UID: \"5e00b411-5685-41e4-8570-faa8f022c493\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7" Jul 21 06:46:52.560005 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.559928 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7" Jul 21 06:46:52.898123 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:52.898102 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7"] Jul 21 06:46:52.899484 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:46:52.899460 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e00b411_5685_41e4_8570_faa8f022c493.slice/crio-d966085d84b87e74307732273de2ef3286526764517d64271e31d7dd6647c55b WatchSource:0}: Error finding container d966085d84b87e74307732273de2ef3286526764517d64271e31d7dd6647c55b: Status 404 returned error can't find the container with id d966085d84b87e74307732273de2ef3286526764517d64271e31d7dd6647c55b Jul 21 06:46:53.134255 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:53.134202 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7" event={"ID":"5e00b411-5685-41e4-8570-faa8f022c493","Type":"ContainerStarted","Data":"d966085d84b87e74307732273de2ef3286526764517d64271e31d7dd6647c55b"} Jul 21 06:46:54.141905 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:54.141865 2563 generic.go:356] "Generic (PLEG): container finished" podID="5e00b411-5685-41e4-8570-faa8f022c493" containerID="c9e49d7a40ac9ebc132dcf16368ae362bbacf88582dfbd17872d0824400d58c3" exitCode=0 Jul 21 06:46:54.142325 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:54.141964 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7" event={"ID":"5e00b411-5685-41e4-8570-faa8f022c493","Type":"ContainerDied","Data":"c9e49d7a40ac9ebc132dcf16368ae362bbacf88582dfbd17872d0824400d58c3"} Jul 21 06:46:55.283545 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:55.283519 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7" Jul 21 06:46:55.419378 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:55.419282 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e00b411-5685-41e4-8570-faa8f022c493-maas-api-service-ca\") pod \"5e00b411-5685-41e4-8570-faa8f022c493\" (UID: \"5e00b411-5685-41e4-8570-faa8f022c493\") " Jul 21 06:46:55.419656 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:55.419632 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e00b411-5685-41e4-8570-faa8f022c493-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e00b411-5685-41e4-8570-faa8f022c493" (UID: "5e00b411-5685-41e4-8570-faa8f022c493"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:46:55.520298 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:55.520258 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e00b411-5685-41e4-8570-faa8f022c493-maas-api-service-ca\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:46:56.154710 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:56.154680 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7" Jul 21 06:46:56.154867 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:56.154711 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7" event={"ID":"5e00b411-5685-41e4-8570-faa8f022c493","Type":"ContainerDied","Data":"d966085d84b87e74307732273de2ef3286526764517d64271e31d7dd6647c55b"} Jul 21 06:46:56.154867 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:46:56.154752 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d966085d84b87e74307732273de2ef3286526764517d64271e31d7dd6647c55b" Jul 21 06:47:07.322627 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:07.322594 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7"] Jul 21 06:47:07.326801 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:07.326774 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-584a4a02-mh4q7"] Jul 21 06:47:07.466129 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:07.466094 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz"] Jul 21 06:47:07.466416 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:07.466388 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" podUID="2e883da8-628e-40c2-9345-754fbd347356" containerName="istio-proxy" containerID="cri-o://b9dd537067cb19cc7f98f32296d753c4d2d6efb0d6b1b5398a2d85b411fb11b1" gracePeriod=30 Jul 21 06:47:08.487071 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:08.487037 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e00b411-5685-41e4-8570-faa8f022c493" path="/var/lib/kubelet/pods/5e00b411-5685-41e4-8570-faa8f022c493/volumes" Jul 21 06:47:12.714859 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.714834 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:47:12.786901 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.786828 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-credential-socket\") pod \"2e883da8-628e-40c2-9345-754fbd347356\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " Jul 21 06:47:12.786901 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.786872 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-workload-socket\") pod \"2e883da8-628e-40c2-9345-754fbd347356\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " Jul 21 06:47:12.786901 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.786895 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e883da8-628e-40c2-9345-754fbd347356-istio-podinfo\") pod \"2e883da8-628e-40c2-9345-754fbd347356\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " Jul 21 06:47:12.787160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.786921 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2pkv\" (UniqueName: \"kubernetes.io/projected/2e883da8-628e-40c2-9345-754fbd347356-kube-api-access-d2pkv\") pod \"2e883da8-628e-40c2-9345-754fbd347356\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " Jul 21 06:47:12.787160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.786972 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-istio-envoy\") pod \"2e883da8-628e-40c2-9345-754fbd347356\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " Jul 21 06:47:12.787160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.787026 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e883da8-628e-40c2-9345-754fbd347356-istio-token\") pod \"2e883da8-628e-40c2-9345-754fbd347356\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " Jul 21 06:47:12.787160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.787060 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e883da8-628e-40c2-9345-754fbd347356-istiod-ca-cert\") pod \"2e883da8-628e-40c2-9345-754fbd347356\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " Jul 21 06:47:12.787160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.787093 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-istio-data\") pod \"2e883da8-628e-40c2-9345-754fbd347356\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " Jul 21 06:47:12.787160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.787109 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2e883da8-628e-40c2-9345-754fbd347356" (UID: "2e883da8-628e-40c2-9345-754fbd347356"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:47:12.787160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.787126 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-workload-certs\") pod \"2e883da8-628e-40c2-9345-754fbd347356\" (UID: \"2e883da8-628e-40c2-9345-754fbd347356\") " Jul 21 06:47:12.787160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.787155 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2e883da8-628e-40c2-9345-754fbd347356" (UID: "2e883da8-628e-40c2-9345-754fbd347356"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:47:12.787651 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.787416 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2e883da8-628e-40c2-9345-754fbd347356" (UID: "2e883da8-628e-40c2-9345-754fbd347356"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:47:12.787651 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.787458 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e883da8-628e-40c2-9345-754fbd347356-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2e883da8-628e-40c2-9345-754fbd347356" (UID: "2e883da8-628e-40c2-9345-754fbd347356"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:47:12.787651 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.787499 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-workload-certs\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:47:12.787651 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.787518 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-credential-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:47:12.787651 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.787530 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-workload-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:47:12.788079 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.787959 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-istio-data" (OuterVolumeSpecName: "istio-data") pod "2e883da8-628e-40c2-9345-754fbd347356" (UID: "2e883da8-628e-40c2-9345-754fbd347356"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:47:12.789363 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.789331 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e883da8-628e-40c2-9345-754fbd347356-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2e883da8-628e-40c2-9345-754fbd347356" (UID: "2e883da8-628e-40c2-9345-754fbd347356"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 06:47:12.789363 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.789331 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2e883da8-628e-40c2-9345-754fbd347356" (UID: "2e883da8-628e-40c2-9345-754fbd347356"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:47:12.789541 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.789397 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e883da8-628e-40c2-9345-754fbd347356-kube-api-access-d2pkv" (OuterVolumeSpecName: "kube-api-access-d2pkv") pod "2e883da8-628e-40c2-9345-754fbd347356" (UID: "2e883da8-628e-40c2-9345-754fbd347356"). InnerVolumeSpecName "kube-api-access-d2pkv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:47:12.789604 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.789541 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e883da8-628e-40c2-9345-754fbd347356-istio-token" (OuterVolumeSpecName: "istio-token") pod "2e883da8-628e-40c2-9345-754fbd347356" (UID: "2e883da8-628e-40c2-9345-754fbd347356"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:47:12.888653 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.888617 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-istio-envoy\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:47:12.888653 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.888648 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e883da8-628e-40c2-9345-754fbd347356-istio-token\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:47:12.888653 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.888659 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e883da8-628e-40c2-9345-754fbd347356-istiod-ca-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:47:12.888895 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.888669 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e883da8-628e-40c2-9345-754fbd347356-istio-data\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:47:12.888895 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.888680 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e883da8-628e-40c2-9345-754fbd347356-istio-podinfo\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:47:12.888895 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:12.888691 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2pkv\" (UniqueName: \"kubernetes.io/projected/2e883da8-628e-40c2-9345-754fbd347356-kube-api-access-d2pkv\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:47:13.271123 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:13.271091 2563 generic.go:356] "Generic (PLEG): container finished" podID="2e883da8-628e-40c2-9345-754fbd347356" containerID="b9dd537067cb19cc7f98f32296d753c4d2d6efb0d6b1b5398a2d85b411fb11b1" exitCode=0 Jul 21 06:47:13.271306 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:13.271142 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" Jul 21 06:47:13.271306 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:13.271181 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" event={"ID":"2e883da8-628e-40c2-9345-754fbd347356","Type":"ContainerDied","Data":"b9dd537067cb19cc7f98f32296d753c4d2d6efb0d6b1b5398a2d85b411fb11b1"} Jul 21 06:47:13.271306 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:13.271256 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz" event={"ID":"2e883da8-628e-40c2-9345-754fbd347356","Type":"ContainerDied","Data":"a27b0316de1d3e9bbcb5d89d01ebd5f30c28e277359c1ca433a4a8717fd6eb60"} Jul 21 06:47:13.271306 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:13.271290 2563 scope.go:117] "RemoveContainer" containerID="b9dd537067cb19cc7f98f32296d753c4d2d6efb0d6b1b5398a2d85b411fb11b1" Jul 21 06:47:13.280986 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:13.280971 2563 scope.go:117] "RemoveContainer" containerID="a6e0c1fab36897391f7386f727d0faeedb3468585cb699f633b308efa1be24a8" Jul 21 06:47:13.289142 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:13.289125 2563 scope.go:117] "RemoveContainer" containerID="b9dd537067cb19cc7f98f32296d753c4d2d6efb0d6b1b5398a2d85b411fb11b1" Jul 21 06:47:13.289456 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:47:13.289433 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9dd537067cb19cc7f98f32296d753c4d2d6efb0d6b1b5398a2d85b411fb11b1\": container with ID starting with b9dd537067cb19cc7f98f32296d753c4d2d6efb0d6b1b5398a2d85b411fb11b1 not found: ID does not exist" containerID="b9dd537067cb19cc7f98f32296d753c4d2d6efb0d6b1b5398a2d85b411fb11b1" Jul 21 06:47:13.289537 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:13.289464 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9dd537067cb19cc7f98f32296d753c4d2d6efb0d6b1b5398a2d85b411fb11b1"} err="failed to get container status \"b9dd537067cb19cc7f98f32296d753c4d2d6efb0d6b1b5398a2d85b411fb11b1\": rpc error: code = NotFound desc = could not find container \"b9dd537067cb19cc7f98f32296d753c4d2d6efb0d6b1b5398a2d85b411fb11b1\": container with ID starting with b9dd537067cb19cc7f98f32296d753c4d2d6efb0d6b1b5398a2d85b411fb11b1 not found: ID does not exist" Jul 21 06:47:13.289537 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:13.289484 2563 scope.go:117] "RemoveContainer" containerID="a6e0c1fab36897391f7386f727d0faeedb3468585cb699f633b308efa1be24a8" Jul 21 06:47:13.289749 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:47:13.289733 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6e0c1fab36897391f7386f727d0faeedb3468585cb699f633b308efa1be24a8\": container with ID starting with a6e0c1fab36897391f7386f727d0faeedb3468585cb699f633b308efa1be24a8 not found: ID does not exist" containerID="a6e0c1fab36897391f7386f727d0faeedb3468585cb699f633b308efa1be24a8" Jul 21 06:47:13.289796 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:13.289753 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6e0c1fab36897391f7386f727d0faeedb3468585cb699f633b308efa1be24a8"} err="failed to get container status \"a6e0c1fab36897391f7386f727d0faeedb3468585cb699f633b308efa1be24a8\": rpc error: code = NotFound desc = could not find container \"a6e0c1fab36897391f7386f727d0faeedb3468585cb699f633b308efa1be24a8\": container with ID starting with a6e0c1fab36897391f7386f727d0faeedb3468585cb699f633b308efa1be24a8 not found: ID does not exist" Jul 21 06:47:13.293780 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:13.293758 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz"] Jul 21 06:47:13.297362 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:13.297343 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-584a4a02-openshift-default-6656d744c-qqfkz"] Jul 21 06:47:14.486652 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:47:14.486623 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e883da8-628e-40c2-9345-754fbd347356" path="/var/lib/kubelet/pods/2e883da8-628e-40c2-9345-754fbd347356/volumes" Jul 21 06:48:45.456767 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.456736 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j"] Jul 21 06:48:45.457277 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.457130 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e883da8-628e-40c2-9345-754fbd347356" containerName="istio-proxy" Jul 21 06:48:45.457277 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.457140 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e883da8-628e-40c2-9345-754fbd347356" containerName="istio-proxy" Jul 21 06:48:45.457277 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.457167 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e00b411-5685-41e4-8570-faa8f022c493" containerName="revoke-keys" Jul 21 06:48:45.457277 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.457172 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e00b411-5685-41e4-8570-faa8f022c493" containerName="revoke-keys" Jul 21 06:48:45.457277 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.457250 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e00b411-5685-41e4-8570-faa8f022c493" containerName="revoke-keys" Jul 21 06:48:45.457277 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.457260 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e883da8-628e-40c2-9345-754fbd347356" containerName="istio-proxy" Jul 21 06:48:45.457277 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.457268 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e883da8-628e-40c2-9345-754fbd347356" containerName="istio-proxy" Jul 21 06:48:45.460345 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.460327 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.463406 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.463383 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e51ee93a-openshift-default-dockercfg-54244\"" Jul 21 06:48:45.486258 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.486232 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j"] Jul 21 06:48:45.554232 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.554177 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de792c58-f06f-4028-be67-c1f213c85246-istiod-ca-cert\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.554422 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.554246 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtkl4\" (UniqueName: \"kubernetes.io/projected/de792c58-f06f-4028-be67-c1f213c85246-kube-api-access-qtkl4\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.554422 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.554293 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-workload-socket\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.554422 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.554322 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-istio-envoy\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.554422 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.554371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de792c58-f06f-4028-be67-c1f213c85246-istio-token\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.554647 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.554441 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-istio-data\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.554647 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.554492 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-workload-certs\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.554647 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.554529 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de792c58-f06f-4028-be67-c1f213c85246-istio-podinfo\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.554647 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.554558 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-credential-socket\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.655431 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.655395 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-workload-socket\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.655431 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.655436 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-istio-envoy\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.655684 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.655472 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de792c58-f06f-4028-be67-c1f213c85246-istio-token\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.655684 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.655541 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-istio-data\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.655684 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.655582 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-workload-certs\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.655684 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.655614 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de792c58-f06f-4028-be67-c1f213c85246-istio-podinfo\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.655684 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.655638 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-credential-socket\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.655994 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.655816 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-workload-socket\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.655994 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.655909 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-workload-certs\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.655994 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.655939 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de792c58-f06f-4028-be67-c1f213c85246-istiod-ca-cert\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.655994 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.655969 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtkl4\" (UniqueName: \"kubernetes.io/projected/de792c58-f06f-4028-be67-c1f213c85246-kube-api-access-qtkl4\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.655994 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.655975 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-credential-socket\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.656854 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.656825 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-istio-data\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.657505 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.657448 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de792c58-f06f-4028-be67-c1f213c85246-istiod-ca-cert\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.664859 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.664829 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-istio-envoy\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.665183 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.665152 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de792c58-f06f-4028-be67-c1f213c85246-istio-podinfo\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.667412 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.667388 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de792c58-f06f-4028-be67-c1f213c85246-istio-token\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.667895 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.667862 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtkl4\" (UniqueName: \"kubernetes.io/projected/de792c58-f06f-4028-be67-c1f213c85246-kube-api-access-qtkl4\") pod \"e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.770747 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.770668 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:45.917980 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.917956 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j"] Jul 21 06:48:45.919312 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:48:45.919285 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde792c58_f06f_4028_be67_c1f213c85246.slice/crio-0ccd462d4d616819327d98fafa941feab2fbcc3a320a0008ab7e462533d3486c WatchSource:0}: Error finding container 0ccd462d4d616819327d98fafa941feab2fbcc3a320a0008ab7e462533d3486c: Status 404 returned error can't find the container with id 0ccd462d4d616819327d98fafa941feab2fbcc3a320a0008ab7e462533d3486c Jul 21 06:48:45.921863 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.921829 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:48:45.921959 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.921893 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:48:45.921959 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:45.921920 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:48:46.809840 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:46.809806 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" event={"ID":"de792c58-f06f-4028-be67-c1f213c85246","Type":"ContainerStarted","Data":"b6191ac299aa9421c9cd7ea1549d138718ec8fc129c84d9814d0cf7350c23779"} Jul 21 06:48:46.810255 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:46.809859 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" event={"ID":"de792c58-f06f-4028-be67-c1f213c85246","Type":"ContainerStarted","Data":"0ccd462d4d616819327d98fafa941feab2fbcc3a320a0008ab7e462533d3486c"} Jul 21 06:48:46.840226 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:46.840171 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" podStartSLOduration=1.84015724 podStartE2EDuration="1.84015724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:48:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:48:46.836617223 +0000 UTC m=+1856.782255881" watchObservedRunningTime="2026-07-21 06:48:46.84015724 +0000 UTC m=+1856.785795897" Jul 21 06:48:47.771178 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:47.771139 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:47.775594 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:47.775569 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:47.816066 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:47.816040 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:48:47.816896 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:48:47.816877 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:49:00.761713 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:00.761646 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx"] Jul 21 06:49:00.762390 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:00.762366 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e883da8-628e-40c2-9345-754fbd347356" containerName="istio-proxy" Jul 21 06:49:00.762478 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:00.762392 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e883da8-628e-40c2-9345-754fbd347356" containerName="istio-proxy" Jul 21 06:49:00.766363 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:00.766345 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx" Jul 21 06:49:00.770135 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:00.770108 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jvhnq\"" Jul 21 06:49:00.771076 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:00.771055 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 06:49:00.772910 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:00.772889 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx"] Jul 21 06:49:00.910110 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:00.910075 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29a82253-ff89-48cc-8347-c2fe7b13f92c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx\" (UID: \"29a82253-ff89-48cc-8347-c2fe7b13f92c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx" Jul 21 06:49:01.011700 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:01.011665 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29a82253-ff89-48cc-8347-c2fe7b13f92c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx\" (UID: \"29a82253-ff89-48cc-8347-c2fe7b13f92c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx" Jul 21 06:49:01.012372 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:01.012311 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29a82253-ff89-48cc-8347-c2fe7b13f92c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx\" (UID: \"29a82253-ff89-48cc-8347-c2fe7b13f92c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx" Jul 21 06:49:01.079856 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:01.079827 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx" Jul 21 06:49:01.223935 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:49:01.223905 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29a82253_ff89_48cc_8347_c2fe7b13f92c.slice/crio-73d1ca1e163ce930cf9357bc8a71b592fb08c37e5d436ef5f4ca2ecd528fdc98 WatchSource:0}: Error finding container 73d1ca1e163ce930cf9357bc8a71b592fb08c37e5d436ef5f4ca2ecd528fdc98: Status 404 returned error can't find the container with id 73d1ca1e163ce930cf9357bc8a71b592fb08c37e5d436ef5f4ca2ecd528fdc98 Jul 21 06:49:01.224357 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:01.224329 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx"] Jul 21 06:49:01.901112 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:01.901071 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx" event={"ID":"29a82253-ff89-48cc-8347-c2fe7b13f92c","Type":"ContainerStarted","Data":"1c3d3fae586fd97a42830f8403230b99a3c6394835a8effc38309dced6aa716c"} Jul 21 06:49:01.901575 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:01.901182 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx" event={"ID":"29a82253-ff89-48cc-8347-c2fe7b13f92c","Type":"ContainerStarted","Data":"73d1ca1e163ce930cf9357bc8a71b592fb08c37e5d436ef5f4ca2ecd528fdc98"} Jul 21 06:49:01.917721 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:01.917670 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx" podStartSLOduration=1.9176544359999999 podStartE2EDuration="1.917654436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:49:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:49:01.916348379 +0000 UTC m=+1871.861987036" watchObservedRunningTime="2026-07-21 06:49:01.917654436 +0000 UTC m=+1871.863293092" Jul 21 06:49:02.908123 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:02.908078 2563 generic.go:356] "Generic (PLEG): container finished" podID="29a82253-ff89-48cc-8347-c2fe7b13f92c" containerID="1c3d3fae586fd97a42830f8403230b99a3c6394835a8effc38309dced6aa716c" exitCode=0 Jul 21 06:49:02.908588 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:02.908148 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx" event={"ID":"29a82253-ff89-48cc-8347-c2fe7b13f92c","Type":"ContainerDied","Data":"1c3d3fae586fd97a42830f8403230b99a3c6394835a8effc38309dced6aa716c"} Jul 21 06:49:03.857608 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:03.857343 2563 patch_prober.go:28] interesting pod/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 06:49:03.857608 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:03.857410 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" podUID="de792c58-f06f-4028-be67-c1f213c85246" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 06:49:04.067988 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:04.067961 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx" Jul 21 06:49:04.243645 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:04.243546 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29a82253-ff89-48cc-8347-c2fe7b13f92c-maas-api-service-ca\") pod \"29a82253-ff89-48cc-8347-c2fe7b13f92c\" (UID: \"29a82253-ff89-48cc-8347-c2fe7b13f92c\") " Jul 21 06:49:04.244002 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:04.243976 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29a82253-ff89-48cc-8347-c2fe7b13f92c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29a82253-ff89-48cc-8347-c2fe7b13f92c" (UID: "29a82253-ff89-48cc-8347-c2fe7b13f92c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:49:04.345002 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:04.344967 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29a82253-ff89-48cc-8347-c2fe7b13f92c-maas-api-service-ca\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:49:04.921506 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:04.921475 2563 generic.go:356] "Generic (PLEG): container finished" podID="de792c58-f06f-4028-be67-c1f213c85246" containerID="b6191ac299aa9421c9cd7ea1549d138718ec8fc129c84d9814d0cf7350c23779" exitCode=0 Jul 21 06:49:04.921686 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:04.921547 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" event={"ID":"de792c58-f06f-4028-be67-c1f213c85246","Type":"ContainerDied","Data":"b6191ac299aa9421c9cd7ea1549d138718ec8fc129c84d9814d0cf7350c23779"} Jul 21 06:49:04.922037 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:04.922017 2563 scope.go:117] "RemoveContainer" containerID="b6191ac299aa9421c9cd7ea1549d138718ec8fc129c84d9814d0cf7350c23779" Jul 21 06:49:04.922941 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:04.922921 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx" Jul 21 06:49:04.923027 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:04.922925 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx" event={"ID":"29a82253-ff89-48cc-8347-c2fe7b13f92c","Type":"ContainerDied","Data":"73d1ca1e163ce930cf9357bc8a71b592fb08c37e5d436ef5f4ca2ecd528fdc98"} Jul 21 06:49:04.923087 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:04.923055 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73d1ca1e163ce930cf9357bc8a71b592fb08c37e5d436ef5f4ca2ecd528fdc98" Jul 21 06:49:04.923129 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:04.923078 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:49:04.923164 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:04.923148 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:49:04.923197 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:04.923179 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:49:05.931068 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:05.931029 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" event={"ID":"de792c58-f06f-4028-be67-c1f213c85246","Type":"ContainerStarted","Data":"83629ae41a52daa0762c51109df739e46c94206092a9a46f94f460b6056ee5c9"} Jul 21 06:49:06.771585 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:06.771545 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:49:06.771795 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:06.771771 2563 patch_prober.go:28] interesting pod/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 21 06:49:06.771842 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:06.771826 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" podUID="de792c58-f06f-4028-be67-c1f213c85246" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 21 06:49:07.771642 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:07.771603 2563 patch_prober.go:28] interesting pod/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 21 06:49:07.772016 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:07.771661 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" podUID="de792c58-f06f-4028-be67-c1f213c85246" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 21 06:49:08.771541 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:08.771507 2563 patch_prober.go:28] interesting pod/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 21 06:49:08.771723 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:08.771563 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" podUID="de792c58-f06f-4028-be67-c1f213c85246" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 21 06:49:09.775428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:09.775395 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:49:09.775880 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:09.775842 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:49:09.777022 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:09.777005 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:49:15.900999 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:15.900916 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx"] Jul 21 06:49:15.910619 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:15.910589 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e51ee93a-lhdlx"] Jul 21 06:49:16.486622 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:16.486585 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29a82253-ff89-48cc-8347-c2fe7b13f92c" path="/var/lib/kubelet/pods/29a82253-ff89-48cc-8347-c2fe7b13f92c/volumes" Jul 21 06:49:22.301515 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:22.301482 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j"] Jul 21 06:49:22.301917 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:22.301758 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" podUID="de792c58-f06f-4028-be67-c1f213c85246" containerName="istio-proxy" containerID="cri-o://83629ae41a52daa0762c51109df739e46c94206092a9a46f94f460b6056ee5c9" gracePeriod=30 Jul 21 06:49:24.778638 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:24.778603 2563 patch_prober.go:28] interesting pod/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 21 06:49:24.779016 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:24.778665 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" podUID="de792c58-f06f-4028-be67-c1f213c85246" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 06:49:27.564795 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.564736 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:49:27.669586 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.669554 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-istio-data\") pod \"de792c58-f06f-4028-be67-c1f213c85246\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " Jul 21 06:49:27.669751 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.669622 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtkl4\" (UniqueName: \"kubernetes.io/projected/de792c58-f06f-4028-be67-c1f213c85246-kube-api-access-qtkl4\") pod \"de792c58-f06f-4028-be67-c1f213c85246\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " Jul 21 06:49:27.669751 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.669675 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-credential-socket\") pod \"de792c58-f06f-4028-be67-c1f213c85246\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " Jul 21 06:49:27.669751 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.669717 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de792c58-f06f-4028-be67-c1f213c85246-istiod-ca-cert\") pod \"de792c58-f06f-4028-be67-c1f213c85246\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " Jul 21 06:49:27.669904 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.669767 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-workload-socket\") pod \"de792c58-f06f-4028-be67-c1f213c85246\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " Jul 21 06:49:27.669904 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.669832 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de792c58-f06f-4028-be67-c1f213c85246-istio-token\") pod \"de792c58-f06f-4028-be67-c1f213c85246\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " Jul 21 06:49:27.669904 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.669863 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-workload-certs\") pod \"de792c58-f06f-4028-be67-c1f213c85246\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " Jul 21 06:49:27.669904 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.669898 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de792c58-f06f-4028-be67-c1f213c85246-istio-podinfo\") pod \"de792c58-f06f-4028-be67-c1f213c85246\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " Jul 21 06:49:27.670109 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.669939 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-istio-envoy\") pod \"de792c58-f06f-4028-be67-c1f213c85246\" (UID: \"de792c58-f06f-4028-be67-c1f213c85246\") " Jul 21 06:49:27.670109 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.670038 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "de792c58-f06f-4028-be67-c1f213c85246" (UID: "de792c58-f06f-4028-be67-c1f213c85246"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:49:27.670244 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.670184 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "de792c58-f06f-4028-be67-c1f213c85246" (UID: "de792c58-f06f-4028-be67-c1f213c85246"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:49:27.670311 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.670281 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "de792c58-f06f-4028-be67-c1f213c85246" (UID: "de792c58-f06f-4028-be67-c1f213c85246"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:49:27.670479 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.670459 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-istio-data" (OuterVolumeSpecName: "istio-data") pod "de792c58-f06f-4028-be67-c1f213c85246" (UID: "de792c58-f06f-4028-be67-c1f213c85246"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:49:27.670568 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.670497 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de792c58-f06f-4028-be67-c1f213c85246-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "de792c58-f06f-4028-be67-c1f213c85246" (UID: "de792c58-f06f-4028-be67-c1f213c85246"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:49:27.670568 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.670474 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-credential-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:49:27.670845 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.670582 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-workload-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:49:27.670845 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.670596 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-workload-certs\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:49:27.672135 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.672101 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de792c58-f06f-4028-be67-c1f213c85246-istio-token" (OuterVolumeSpecName: "istio-token") pod "de792c58-f06f-4028-be67-c1f213c85246" (UID: "de792c58-f06f-4028-be67-c1f213c85246"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:49:27.672135 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.672113 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "de792c58-f06f-4028-be67-c1f213c85246" (UID: "de792c58-f06f-4028-be67-c1f213c85246"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:49:27.672135 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.672120 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de792c58-f06f-4028-be67-c1f213c85246-kube-api-access-qtkl4" (OuterVolumeSpecName: "kube-api-access-qtkl4") pod "de792c58-f06f-4028-be67-c1f213c85246" (UID: "de792c58-f06f-4028-be67-c1f213c85246"). InnerVolumeSpecName "kube-api-access-qtkl4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:49:27.672406 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.672174 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/de792c58-f06f-4028-be67-c1f213c85246-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "de792c58-f06f-4028-be67-c1f213c85246" (UID: "de792c58-f06f-4028-be67-c1f213c85246"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 06:49:27.771904 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.771868 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de792c58-f06f-4028-be67-c1f213c85246-istiod-ca-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:49:27.771904 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.771899 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de792c58-f06f-4028-be67-c1f213c85246-istio-token\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:49:27.771904 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.771908 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de792c58-f06f-4028-be67-c1f213c85246-istio-podinfo\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:49:27.771904 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.771915 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-istio-envoy\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:49:27.772175 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.771924 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de792c58-f06f-4028-be67-c1f213c85246-istio-data\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:49:27.772175 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:27.771932 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtkl4\" (UniqueName: \"kubernetes.io/projected/de792c58-f06f-4028-be67-c1f213c85246-kube-api-access-qtkl4\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:49:28.068748 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:28.068714 2563 generic.go:356] "Generic (PLEG): container finished" podID="de792c58-f06f-4028-be67-c1f213c85246" containerID="83629ae41a52daa0762c51109df739e46c94206092a9a46f94f460b6056ee5c9" exitCode=0 Jul 21 06:49:28.068925 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:28.068763 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" Jul 21 06:49:28.068925 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:28.068799 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" event={"ID":"de792c58-f06f-4028-be67-c1f213c85246","Type":"ContainerDied","Data":"83629ae41a52daa0762c51109df739e46c94206092a9a46f94f460b6056ee5c9"} Jul 21 06:49:28.068925 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:28.068853 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j" event={"ID":"de792c58-f06f-4028-be67-c1f213c85246","Type":"ContainerDied","Data":"0ccd462d4d616819327d98fafa941feab2fbcc3a320a0008ab7e462533d3486c"} Jul 21 06:49:28.068925 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:28.068875 2563 scope.go:117] "RemoveContainer" containerID="83629ae41a52daa0762c51109df739e46c94206092a9a46f94f460b6056ee5c9" Jul 21 06:49:28.078685 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:28.078667 2563 scope.go:117] "RemoveContainer" containerID="b6191ac299aa9421c9cd7ea1549d138718ec8fc129c84d9814d0cf7350c23779" Jul 21 06:49:28.087005 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:28.086991 2563 scope.go:117] "RemoveContainer" containerID="83629ae41a52daa0762c51109df739e46c94206092a9a46f94f460b6056ee5c9" Jul 21 06:49:28.087254 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:49:28.087233 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83629ae41a52daa0762c51109df739e46c94206092a9a46f94f460b6056ee5c9\": container with ID starting with 83629ae41a52daa0762c51109df739e46c94206092a9a46f94f460b6056ee5c9 not found: ID does not exist" containerID="83629ae41a52daa0762c51109df739e46c94206092a9a46f94f460b6056ee5c9" Jul 21 06:49:28.087302 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:28.087264 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83629ae41a52daa0762c51109df739e46c94206092a9a46f94f460b6056ee5c9"} err="failed to get container status \"83629ae41a52daa0762c51109df739e46c94206092a9a46f94f460b6056ee5c9\": rpc error: code = NotFound desc = could not find container \"83629ae41a52daa0762c51109df739e46c94206092a9a46f94f460b6056ee5c9\": container with ID starting with 83629ae41a52daa0762c51109df739e46c94206092a9a46f94f460b6056ee5c9 not found: ID does not exist" Jul 21 06:49:28.087302 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:28.087280 2563 scope.go:117] "RemoveContainer" containerID="b6191ac299aa9421c9cd7ea1549d138718ec8fc129c84d9814d0cf7350c23779" Jul 21 06:49:28.087491 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:49:28.087475 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6191ac299aa9421c9cd7ea1549d138718ec8fc129c84d9814d0cf7350c23779\": container with ID starting with b6191ac299aa9421c9cd7ea1549d138718ec8fc129c84d9814d0cf7350c23779 not found: ID does not exist" containerID="b6191ac299aa9421c9cd7ea1549d138718ec8fc129c84d9814d0cf7350c23779" Jul 21 06:49:28.087535 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:28.087496 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6191ac299aa9421c9cd7ea1549d138718ec8fc129c84d9814d0cf7350c23779"} err="failed to get container status \"b6191ac299aa9421c9cd7ea1549d138718ec8fc129c84d9814d0cf7350c23779\": rpc error: code = NotFound desc = could not find container \"b6191ac299aa9421c9cd7ea1549d138718ec8fc129c84d9814d0cf7350c23779\": container with ID starting with b6191ac299aa9421c9cd7ea1549d138718ec8fc129c84d9814d0cf7350c23779 not found: ID does not exist" Jul 21 06:49:28.094211 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:28.094188 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j"] Jul 21 06:49:28.098978 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:28.098955 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e51ee93a-openshift-default-97f74b56c-kb87j"] Jul 21 06:49:28.486797 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:28.486766 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de792c58-f06f-4028-be67-c1f213c85246" path="/var/lib/kubelet/pods/de792c58-f06f-4028-be67-c1f213c85246/volumes" Jul 21 06:49:44.812040 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.812009 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j"] Jul 21 06:49:44.812454 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.812440 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de792c58-f06f-4028-be67-c1f213c85246" containerName="istio-proxy" Jul 21 06:49:44.812499 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.812455 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="de792c58-f06f-4028-be67-c1f213c85246" containerName="istio-proxy" Jul 21 06:49:44.812499 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.812467 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de792c58-f06f-4028-be67-c1f213c85246" containerName="istio-proxy" Jul 21 06:49:44.812499 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.812473 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="de792c58-f06f-4028-be67-c1f213c85246" containerName="istio-proxy" Jul 21 06:49:44.812499 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.812496 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a82253-ff89-48cc-8347-c2fe7b13f92c" containerName="revoke-keys" Jul 21 06:49:44.812621 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.812502 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a82253-ff89-48cc-8347-c2fe7b13f92c" containerName="revoke-keys" Jul 21 06:49:44.812621 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.812563 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="de792c58-f06f-4028-be67-c1f213c85246" containerName="istio-proxy" Jul 21 06:49:44.812621 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.812572 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a82253-ff89-48cc-8347-c2fe7b13f92c" containerName="revoke-keys" Jul 21 06:49:44.816919 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.816901 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.820313 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.820292 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1ae2daad-openshift-default-dockercfg-brhv2\"" Jul 21 06:49:44.829562 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.829540 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j"] Jul 21 06:49:44.835870 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.835840 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-podinfo\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.835996 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.835903 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-credential-socket\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.835996 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.835943 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-envoy\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.836093 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.835995 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-data\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.836093 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.836033 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istiod-ca-cert\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.836093 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.836057 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gws7s\" (UniqueName: \"kubernetes.io/projected/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-kube-api-access-gws7s\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.836252 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.836137 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-workload-socket\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.836252 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.836190 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-token\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.836368 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.836280 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-workload-certs\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.937514 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.937475 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-podinfo\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.937723 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.937541 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-credential-socket\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.937723 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.937572 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-envoy\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.937723 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.937624 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-data\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.937723 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.937665 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istiod-ca-cert\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.937723 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.937690 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gws7s\" (UniqueName: \"kubernetes.io/projected/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-kube-api-access-gws7s\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.937723 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.937723 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-workload-socket\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.938094 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.937772 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-token\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.938094 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.937817 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-workload-certs\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.938453 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.938192 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-workload-socket\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.938453 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.938210 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-workload-certs\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.938621 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.938503 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-data\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.938678 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.938627 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-credential-socket\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.938717 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.938676 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istiod-ca-cert\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.940078 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.940058 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-envoy\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.940931 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.940906 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-podinfo\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.947809 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.947781 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gws7s\" (UniqueName: \"kubernetes.io/projected/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-kube-api-access-gws7s\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:44.947891 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:44.947811 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-token\") pod \"e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:45.128519 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:45.128486 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:45.272173 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:45.272145 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j"] Jul 21 06:49:45.273632 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:49:45.273596 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f8b0409_d335_4e3a_a3c0_52c6ad473d48.slice/crio-4578e0ab3c1fe33989efc0ea53ac596c93368a57edd5b4b77faa014ccb25b5e9 WatchSource:0}: Error finding container 4578e0ab3c1fe33989efc0ea53ac596c93368a57edd5b4b77faa014ccb25b5e9: Status 404 returned error can't find the container with id 4578e0ab3c1fe33989efc0ea53ac596c93368a57edd5b4b77faa014ccb25b5e9 Jul 21 06:49:45.275808 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:45.275776 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:49:45.275894 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:45.275855 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:49:45.275965 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:45.275901 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:49:46.183211 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:46.183175 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" event={"ID":"8f8b0409-d335-4e3a-a3c0-52c6ad473d48","Type":"ContainerStarted","Data":"b1d3af9bad1e7b8dc4d1b63a26dedee640e41be218c37b991b57f29b864df0d6"} Jul 21 06:49:46.183616 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:46.183247 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" event={"ID":"8f8b0409-d335-4e3a-a3c0-52c6ad473d48","Type":"ContainerStarted","Data":"4578e0ab3c1fe33989efc0ea53ac596c93368a57edd5b4b77faa014ccb25b5e9"} Jul 21 06:49:46.210853 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:46.210806 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" podStartSLOduration=2.21079249 podStartE2EDuration="2.21079249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:49:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:49:46.207160474 +0000 UTC m=+1916.152799148" watchObservedRunningTime="2026-07-21 06:49:46.21079249 +0000 UTC m=+1916.156431146" Jul 21 06:49:47.128919 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:47.128884 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:47.133811 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:47.133787 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:47.188973 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:47.188938 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:49:47.189945 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:49:47.189926 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:50:02.191177 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:02.191148 2563 patch_prober.go:28] interesting pod/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 21 06:50:02.191567 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:02.191238 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 21 06:50:02.288170 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:02.288131 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerID="b1d3af9bad1e7b8dc4d1b63a26dedee640e41be218c37b991b57f29b864df0d6" exitCode=0 Jul 21 06:50:02.288402 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:02.288182 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" event={"ID":"8f8b0409-d335-4e3a-a3c0-52c6ad473d48","Type":"ContainerDied","Data":"b1d3af9bad1e7b8dc4d1b63a26dedee640e41be218c37b991b57f29b864df0d6"} Jul 21 06:50:02.288697 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:02.288679 2563 scope.go:117] "RemoveContainer" containerID="b1d3af9bad1e7b8dc4d1b63a26dedee640e41be218c37b991b57f29b864df0d6" Jul 21 06:50:02.289734 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:02.289707 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:50:02.289799 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:02.289776 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:50:02.289840 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:02.289819 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:50:03.295687 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:03.295653 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" event={"ID":"8f8b0409-d335-4e3a-a3c0-52c6ad473d48","Type":"ContainerStarted","Data":"c39c930b59deb30171a4aea4fe9d61079801bfea650a6d0afdd3a256f652cd66"} Jul 21 06:50:04.129095 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:04.129052 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:50:04.129350 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:04.129329 2563 patch_prober.go:28] interesting pod/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 21 06:50:04.129410 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:04.129376 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 21 06:50:05.128992 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:05.128953 2563 patch_prober.go:28] interesting pod/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 21 06:50:05.129434 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:05.129013 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 21 06:50:06.129529 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:06.129489 2563 patch_prober.go:28] interesting pod/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 21 06:50:06.129951 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:06.129561 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 21 06:50:07.129166 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:07.129129 2563 patch_prober.go:28] interesting pod/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 21 06:50:07.129369 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:07.129183 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 21 06:50:08.133411 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:08.133376 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:50:08.133899 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:08.133724 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:50:08.134575 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:08.134551 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:50:36.490303 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:36.487146 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr"] Jul 21 06:50:36.490303 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:36.488330 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="de792c58-f06f-4028-be67-c1f213c85246" containerName="istio-proxy" Jul 21 06:50:36.491591 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:36.491564 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr" Jul 21 06:50:36.496150 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:36.496125 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jvhnq\"" Jul 21 06:50:36.496439 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:36.496422 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 06:50:36.503388 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:36.503368 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr"] Jul 21 06:50:36.546157 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:36.546126 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0abea6b8-4aa1-42a1-bae2-917a02d85aff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr\" (UID: \"0abea6b8-4aa1-42a1-bae2-917a02d85aff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr" Jul 21 06:50:36.647181 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:36.647148 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0abea6b8-4aa1-42a1-bae2-917a02d85aff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr\" (UID: \"0abea6b8-4aa1-42a1-bae2-917a02d85aff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr" Jul 21 06:50:36.647790 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:36.647770 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0abea6b8-4aa1-42a1-bae2-917a02d85aff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr\" (UID: \"0abea6b8-4aa1-42a1-bae2-917a02d85aff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr" Jul 21 06:50:36.805285 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:36.805191 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr" Jul 21 06:50:36.947462 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:36.947428 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr"] Jul 21 06:50:36.950702 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:50:36.950673 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0abea6b8_4aa1_42a1_bae2_917a02d85aff.slice/crio-bdb51dbccf2b66d3c7ae3b0db080936cf98e398dd3c897cda2046d3c8230783f WatchSource:0}: Error finding container bdb51dbccf2b66d3c7ae3b0db080936cf98e398dd3c897cda2046d3c8230783f: Status 404 returned error can't find the container with id bdb51dbccf2b66d3c7ae3b0db080936cf98e398dd3c897cda2046d3c8230783f Jul 21 06:50:37.489899 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:37.489803 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr" event={"ID":"0abea6b8-4aa1-42a1-bae2-917a02d85aff","Type":"ContainerStarted","Data":"c4e0fb3df36de3d47c7a0af5821738f2dc85fd2c45b3f9103fe0c59971961f03"} Jul 21 06:50:37.489899 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:37.489863 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr" event={"ID":"0abea6b8-4aa1-42a1-bae2-917a02d85aff","Type":"ContainerStarted","Data":"bdb51dbccf2b66d3c7ae3b0db080936cf98e398dd3c897cda2046d3c8230783f"} Jul 21 06:50:37.508919 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:37.508868 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr" podStartSLOduration=1.508854426 podStartE2EDuration="1.508854426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:50:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:50:37.507099838 +0000 UTC m=+1967.452738497" watchObservedRunningTime="2026-07-21 06:50:37.508854426 +0000 UTC m=+1967.454493083" Jul 21 06:50:38.496287 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:38.496251 2563 generic.go:356] "Generic (PLEG): container finished" podID="0abea6b8-4aa1-42a1-bae2-917a02d85aff" containerID="c4e0fb3df36de3d47c7a0af5821738f2dc85fd2c45b3f9103fe0c59971961f03" exitCode=0 Jul 21 06:50:38.496456 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:38.496318 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr" event={"ID":"0abea6b8-4aa1-42a1-bae2-917a02d85aff","Type":"ContainerDied","Data":"c4e0fb3df36de3d47c7a0af5821738f2dc85fd2c45b3f9103fe0c59971961f03"} Jul 21 06:50:39.652983 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:39.652960 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr" Jul 21 06:50:39.779996 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:39.779909 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0abea6b8-4aa1-42a1-bae2-917a02d85aff-maas-api-service-ca\") pod \"0abea6b8-4aa1-42a1-bae2-917a02d85aff\" (UID: \"0abea6b8-4aa1-42a1-bae2-917a02d85aff\") " Jul 21 06:50:39.780301 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:39.780278 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0abea6b8-4aa1-42a1-bae2-917a02d85aff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0abea6b8-4aa1-42a1-bae2-917a02d85aff" (UID: "0abea6b8-4aa1-42a1-bae2-917a02d85aff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:50:39.880621 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:39.880583 2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0abea6b8-4aa1-42a1-bae2-917a02d85aff-maas-api-service-ca\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:50:40.509303 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:40.509275 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr" Jul 21 06:50:40.509444 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:40.509312 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr" event={"ID":"0abea6b8-4aa1-42a1-bae2-917a02d85aff","Type":"ContainerDied","Data":"bdb51dbccf2b66d3c7ae3b0db080936cf98e398dd3c897cda2046d3c8230783f"} Jul 21 06:50:40.509444 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:40.509355 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdb51dbccf2b66d3c7ae3b0db080936cf98e398dd3c897cda2046d3c8230783f" Jul 21 06:50:50.718464 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:50.718380 2563 scope.go:117] "RemoveContainer" containerID="ec0327f46b57c260e37835f9fc216e0532245cb641c65f1fce2e3643ddf06b36" Jul 21 06:50:56.588874 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:56.588841 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr"] Jul 21 06:50:56.596256 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:56.596230 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1ae2daad-gvfvr"] Jul 21 06:50:58.486283 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:50:58.486246 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0abea6b8-4aa1-42a1-bae2-917a02d85aff" path="/var/lib/kubelet/pods/0abea6b8-4aa1-42a1-bae2-917a02d85aff/volumes" Jul 21 06:51:50.741963 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:50.741931 2563 scope.go:117] "RemoveContainer" containerID="e66ab4f1162fef9f2d0c5051de8f9a08ebb9e144038cfbca19758b9a298f830c" Jul 21 06:51:52.094716 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:52.094684 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j"] Jul 21 06:51:52.095175 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:52.094940 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerName="istio-proxy" containerID="cri-o://c39c930b59deb30171a4aea4fe9d61079801bfea650a6d0afdd3a256f652cd66" gracePeriod=30 Jul 21 06:51:53.136288 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:53.136252 2563 patch_prober.go:28] interesting pod/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 06:51:53.136692 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:53.136306 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 06:51:57.358807 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.358748 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:51:57.394766 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.394739 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-credential-socket\") pod \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " Jul 21 06:51:57.394933 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.394777 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-token\") pod \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " Jul 21 06:51:57.394933 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.394794 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-workload-certs\") pod \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " Jul 21 06:51:57.394933 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.394817 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-workload-socket\") pod \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " Jul 21 06:51:57.394933 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.394861 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gws7s\" (UniqueName: \"kubernetes.io/projected/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-kube-api-access-gws7s\") pod \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " Jul 21 06:51:57.394933 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.394894 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istiod-ca-cert\") pod \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " Jul 21 06:51:57.395210 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.394954 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-data\") pod \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " Jul 21 06:51:57.395210 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.394987 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-podinfo\") pod \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " Jul 21 06:51:57.395210 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.395069 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f8b0409-d335-4e3a-a3c0-52c6ad473d48" (UID: "8f8b0409-d335-4e3a-a3c0-52c6ad473d48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:51:57.395210 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.395141 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f8b0409-d335-4e3a-a3c0-52c6ad473d48" (UID: "8f8b0409-d335-4e3a-a3c0-52c6ad473d48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:51:57.395210 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.395211 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-envoy\") pod \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\" (UID: \"8f8b0409-d335-4e3a-a3c0-52c6ad473d48\") " Jul 21 06:51:57.395497 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.395349 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f8b0409-d335-4e3a-a3c0-52c6ad473d48" (UID: "8f8b0409-d335-4e3a-a3c0-52c6ad473d48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:51:57.395572 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.395539 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f8b0409-d335-4e3a-a3c0-52c6ad473d48" (UID: "8f8b0409-d335-4e3a-a3c0-52c6ad473d48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:51:57.395655 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.395639 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-credential-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:51:57.395710 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.395661 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-workload-certs\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:51:57.395710 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.395676 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-workload-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:51:57.395710 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.395688 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istiod-ca-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:51:57.395986 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.395964 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f8b0409-d335-4e3a-a3c0-52c6ad473d48" (UID: "8f8b0409-d335-4e3a-a3c0-52c6ad473d48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:51:57.397561 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.397535 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f8b0409-d335-4e3a-a3c0-52c6ad473d48" (UID: "8f8b0409-d335-4e3a-a3c0-52c6ad473d48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:51:57.397803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.397779 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f8b0409-d335-4e3a-a3c0-52c6ad473d48" (UID: "8f8b0409-d335-4e3a-a3c0-52c6ad473d48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 06:51:57.397873 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.397844 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f8b0409-d335-4e3a-a3c0-52c6ad473d48" (UID: "8f8b0409-d335-4e3a-a3c0-52c6ad473d48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:51:57.397988 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.397971 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-kube-api-access-gws7s" (OuterVolumeSpecName: "kube-api-access-gws7s") pod "8f8b0409-d335-4e3a-a3c0-52c6ad473d48" (UID: "8f8b0409-d335-4e3a-a3c0-52c6ad473d48"). InnerVolumeSpecName "kube-api-access-gws7s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:51:57.496727 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.496698 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-token\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:51:57.496727 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.496727 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gws7s\" (UniqueName: \"kubernetes.io/projected/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-kube-api-access-gws7s\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:51:57.496920 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.496737 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-data\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:51:57.496920 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.496745 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-podinfo\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:51:57.496920 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.496753 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f8b0409-d335-4e3a-a3c0-52c6ad473d48-istio-envoy\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:51:57.959089 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.959054 2563 generic.go:356] "Generic (PLEG): container finished" podID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerID="c39c930b59deb30171a4aea4fe9d61079801bfea650a6d0afdd3a256f652cd66" exitCode=0 Jul 21 06:51:57.959292 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.959104 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" Jul 21 06:51:57.959292 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.959113 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" event={"ID":"8f8b0409-d335-4e3a-a3c0-52c6ad473d48","Type":"ContainerDied","Data":"c39c930b59deb30171a4aea4fe9d61079801bfea650a6d0afdd3a256f652cd66"} Jul 21 06:51:57.959292 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.959163 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j" event={"ID":"8f8b0409-d335-4e3a-a3c0-52c6ad473d48","Type":"ContainerDied","Data":"4578e0ab3c1fe33989efc0ea53ac596c93368a57edd5b4b77faa014ccb25b5e9"} Jul 21 06:51:57.959292 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.959186 2563 scope.go:117] "RemoveContainer" containerID="c39c930b59deb30171a4aea4fe9d61079801bfea650a6d0afdd3a256f652cd66" Jul 21 06:51:57.969752 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.969732 2563 scope.go:117] "RemoveContainer" containerID="b1d3af9bad1e7b8dc4d1b63a26dedee640e41be218c37b991b57f29b864df0d6" Jul 21 06:51:57.978342 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.978325 2563 scope.go:117] "RemoveContainer" containerID="c39c930b59deb30171a4aea4fe9d61079801bfea650a6d0afdd3a256f652cd66" Jul 21 06:51:57.978591 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:51:57.978575 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c39c930b59deb30171a4aea4fe9d61079801bfea650a6d0afdd3a256f652cd66\": container with ID starting with c39c930b59deb30171a4aea4fe9d61079801bfea650a6d0afdd3a256f652cd66 not found: ID does not exist" containerID="c39c930b59deb30171a4aea4fe9d61079801bfea650a6d0afdd3a256f652cd66" Jul 21 06:51:57.978666 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.978599 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c39c930b59deb30171a4aea4fe9d61079801bfea650a6d0afdd3a256f652cd66"} err="failed to get container status \"c39c930b59deb30171a4aea4fe9d61079801bfea650a6d0afdd3a256f652cd66\": rpc error: code = NotFound desc = could not find container \"c39c930b59deb30171a4aea4fe9d61079801bfea650a6d0afdd3a256f652cd66\": container with ID starting with c39c930b59deb30171a4aea4fe9d61079801bfea650a6d0afdd3a256f652cd66 not found: ID does not exist" Jul 21 06:51:57.978666 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.978616 2563 scope.go:117] "RemoveContainer" containerID="b1d3af9bad1e7b8dc4d1b63a26dedee640e41be218c37b991b57f29b864df0d6" Jul 21 06:51:57.978927 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:51:57.978890 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1d3af9bad1e7b8dc4d1b63a26dedee640e41be218c37b991b57f29b864df0d6\": container with ID starting with b1d3af9bad1e7b8dc4d1b63a26dedee640e41be218c37b991b57f29b864df0d6 not found: ID does not exist" containerID="b1d3af9bad1e7b8dc4d1b63a26dedee640e41be218c37b991b57f29b864df0d6" Jul 21 06:51:57.979015 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.978931 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1d3af9bad1e7b8dc4d1b63a26dedee640e41be218c37b991b57f29b864df0d6"} err="failed to get container status \"b1d3af9bad1e7b8dc4d1b63a26dedee640e41be218c37b991b57f29b864df0d6\": rpc error: code = NotFound desc = could not find container \"b1d3af9bad1e7b8dc4d1b63a26dedee640e41be218c37b991b57f29b864df0d6\": container with ID starting with b1d3af9bad1e7b8dc4d1b63a26dedee640e41be218c37b991b57f29b864df0d6 not found: ID does not exist" Jul 21 06:51:57.982823 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.982802 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j"] Jul 21 06:51:57.988156 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:57.988133 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1ae2daad-openshift-default-5bc4d65dff-sjq4j"] Jul 21 06:51:58.490098 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:51:58.490070 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" path="/var/lib/kubelet/pods/8f8b0409-d335-4e3a-a3c0-52c6ad473d48/volumes" Jul 21 06:52:48.592935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.592897 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6"] Jul 21 06:52:48.593527 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.593336 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerName="istio-proxy" Jul 21 06:52:48.593527 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.593354 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerName="istio-proxy" Jul 21 06:52:48.593527 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.593369 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerName="istio-proxy" Jul 21 06:52:48.593527 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.593375 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerName="istio-proxy" Jul 21 06:52:48.593527 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.593382 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0abea6b8-4aa1-42a1-bae2-917a02d85aff" containerName="revoke-keys" Jul 21 06:52:48.593527 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.593388 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0abea6b8-4aa1-42a1-bae2-917a02d85aff" containerName="revoke-keys" Jul 21 06:52:48.593527 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.593487 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0abea6b8-4aa1-42a1-bae2-917a02d85aff" containerName="revoke-keys" Jul 21 06:52:48.593527 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.593496 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerName="istio-proxy" Jul 21 06:52:48.593527 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.593502 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f8b0409-d335-4e3a-a3c0-52c6ad473d48" containerName="istio-proxy" Jul 21 06:52:48.596800 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.596778 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.601302 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.601277 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6d945c-openshift-default-dockercfg-2sw8b\"" Jul 21 06:52:48.617610 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.617585 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6"] Jul 21 06:52:48.675198 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.675159 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-envoy\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.675395 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.675288 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-token\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.675395 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.675346 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-credential-socket\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.675395 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.675381 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-workload-certs\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.675539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.675472 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-workload-socket\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.675539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.675511 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istiod-ca-cert\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.675539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.675536 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2ldp\" (UniqueName: \"kubernetes.io/projected/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-kube-api-access-g2ldp\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.675656 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.675575 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-data\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.675656 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.675607 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-podinfo\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.776699 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.776662 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-workload-certs\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.776699 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.776703 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-workload-socket\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.776955 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.776728 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istiod-ca-cert\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.776955 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.776752 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2ldp\" (UniqueName: \"kubernetes.io/projected/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-kube-api-access-g2ldp\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.776955 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.776774 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-data\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.776955 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.776914 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-podinfo\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.777162 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.776967 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-envoy\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.777162 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.777073 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-token\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.777162 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.777144 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-credential-socket\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.777344 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.777171 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-data\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.777344 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.777203 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-workload-socket\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.777555 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.777535 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-workload-certs\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.777632 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.777536 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-credential-socket\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.777632 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.777540 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istiod-ca-cert\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.779532 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.779509 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-envoy\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.779667 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.779649 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-podinfo\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.787457 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.787423 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-token\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.787585 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.787552 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2ldp\" (UniqueName: \"kubernetes.io/projected/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-kube-api-access-g2ldp\") pod \"e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:48.909142 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:48.909062 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:49.053813 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:49.053787 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6"] Jul 21 06:52:49.055073 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:52:49.055041 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ce8023_f33e_4dd5_a9ab_4b317a958f4c.slice/crio-50e695873e4d082f88b4c39838275604d7e74f96b50c813e07c2558877e01b3c WatchSource:0}: Error finding container 50e695873e4d082f88b4c39838275604d7e74f96b50c813e07c2558877e01b3c: Status 404 returned error can't find the container with id 50e695873e4d082f88b4c39838275604d7e74f96b50c813e07c2558877e01b3c Jul 21 06:52:49.057181 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:49.057149 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:52:49.057302 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:49.057251 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:52:49.057371 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:49.057300 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:52:49.267099 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:49.267000 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" event={"ID":"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c","Type":"ContainerStarted","Data":"71e5ab8e0ad8a4a2f49c212f2536b26e82351a94d71f771328d39b620fb967ba"} Jul 21 06:52:49.267099 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:49.267065 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" event={"ID":"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c","Type":"ContainerStarted","Data":"50e695873e4d082f88b4c39838275604d7e74f96b50c813e07c2558877e01b3c"} Jul 21 06:52:49.287529 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:49.287483 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" podStartSLOduration=1.287467369 podStartE2EDuration="1.287467369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:52:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:52:49.286805439 +0000 UTC m=+2099.232444097" watchObservedRunningTime="2026-07-21 06:52:49.287467369 +0000 UTC m=+2099.233106028" Jul 21 06:52:49.909718 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:49.909677 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:50.914659 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:50.914629 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:51.279194 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:51.279163 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:51.280208 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:51.280186 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:52:56.439486 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.439455 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt"] Jul 21 06:52:56.443135 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.443111 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.447060 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.447033 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-11b68c-openshift-default-dockercfg-249pf\"" Jul 21 06:52:56.466856 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.466829 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt"] Jul 21 06:52:56.564317 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.564281 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-envoy\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.564495 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.564326 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istiod-ca-cert\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.564495 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.564347 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-token\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.564495 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.564371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-data\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.564495 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.564440 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-workload-certs\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.564704 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.564653 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-podinfo\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.564899 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.564876 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-credential-socket\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.564967 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.564950 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqzqf\" (UniqueName: \"kubernetes.io/projected/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-kube-api-access-tqzqf\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.565006 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.564979 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-workload-socket\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.666053 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.666015 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-workload-certs\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.666266 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.666104 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-podinfo\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.666266 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.666134 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-credential-socket\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.666266 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.666173 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqzqf\" (UniqueName: \"kubernetes.io/projected/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-kube-api-access-tqzqf\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.666446 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.666294 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-workload-socket\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.666497 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.666465 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-workload-certs\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.666552 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.666514 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-envoy\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.666552 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.666529 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-credential-socket\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.666661 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.666597 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istiod-ca-cert\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.666661 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.666629 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-token\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.666756 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.666658 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-data\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.666756 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.666678 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-workload-socket\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.666934 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.666913 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-data\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.667387 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.667366 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istiod-ca-cert\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.668632 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.668610 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-envoy\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.668739 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.668718 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-podinfo\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.674604 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.674579 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-token\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.674796 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.674777 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqzqf\" (UniqueName: \"kubernetes.io/projected/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-kube-api-access-tqzqf\") pod \"e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.753870 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.753787 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:56.896766 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.896736 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt"] Jul 21 06:52:56.897472 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:52:56.897446 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ceafc8_bc72_4e90_8ebb_bf4feedffebf.slice/crio-2dc5cf8bcdb68add85efd4c5e4c334691eca7174f4c71c3a17434f40c6b9877c WatchSource:0}: Error finding container 2dc5cf8bcdb68add85efd4c5e4c334691eca7174f4c71c3a17434f40c6b9877c: Status 404 returned error can't find the container with id 2dc5cf8bcdb68add85efd4c5e4c334691eca7174f4c71c3a17434f40c6b9877c Jul 21 06:52:56.899587 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.899557 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:52:56.899694 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.899629 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:52:56.899694 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:56.899672 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:52:57.320210 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:57.320175 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" event={"ID":"69ceafc8-bc72-4e90-8ebb-bf4feedffebf","Type":"ContainerStarted","Data":"460a6b776731577383445480b0280f4923f6aae3300742af113cb689f6f86f33"} Jul 21 06:52:57.320408 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:57.320248 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" event={"ID":"69ceafc8-bc72-4e90-8ebb-bf4feedffebf","Type":"ContainerStarted","Data":"2dc5cf8bcdb68add85efd4c5e4c334691eca7174f4c71c3a17434f40c6b9877c"} Jul 21 06:52:57.341960 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:57.341896 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" podStartSLOduration=1.341876526 podStartE2EDuration="1.341876526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:52:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:52:57.341160751 +0000 UTC m=+2107.286799409" watchObservedRunningTime="2026-07-21 06:52:57.341876526 +0000 UTC m=+2107.287515187" Jul 21 06:52:57.754258 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:57.754201 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:57.759067 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:57.759042 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:58.326592 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:58.326565 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:52:58.327447 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:52:58.327428 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:53:06.743694 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:06.742581 2563 patch_prober.go:28] interesting pod/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:45858->10.134.0.63:15021: read: connection reset by peer" start-of-body= Jul 21 06:53:06.743694 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:06.742672 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:45858->10.134.0.63:15021: read: connection reset by peer" Jul 21 06:53:07.390751 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:07.390718 2563 generic.go:356] "Generic (PLEG): container finished" podID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerID="71e5ab8e0ad8a4a2f49c212f2536b26e82351a94d71f771328d39b620fb967ba" exitCode=0 Jul 21 06:53:07.390926 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:07.390783 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" event={"ID":"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c","Type":"ContainerDied","Data":"71e5ab8e0ad8a4a2f49c212f2536b26e82351a94d71f771328d39b620fb967ba"} Jul 21 06:53:07.391156 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:07.391141 2563 scope.go:117] "RemoveContainer" containerID="71e5ab8e0ad8a4a2f49c212f2536b26e82351a94d71f771328d39b620fb967ba" Jul 21 06:53:07.392135 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:07.392109 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:53:07.392204 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:07.392174 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:53:07.392256 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:07.392203 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:53:08.400370 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:08.400332 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" event={"ID":"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c","Type":"ContainerStarted","Data":"348cfb5e47404ab364157fc0823755100577bc4608a20c4dff3aaed7aa622015"} Jul 21 06:53:08.909371 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:08.909325 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:53:08.909656 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:08.909636 2563 patch_prober.go:28] interesting pod/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 21 06:53:08.909706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:08.909688 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 21 06:53:09.909800 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:09.909761 2563 patch_prober.go:28] interesting pod/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 21 06:53:09.910252 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:09.909832 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 21 06:53:10.909782 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:10.909744 2563 patch_prober.go:28] interesting pod/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 21 06:53:10.909964 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:10.909803 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 21 06:53:11.909996 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:11.909955 2563 patch_prober.go:28] interesting pod/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 21 06:53:11.910421 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:11.910036 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 21 06:53:12.913711 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:12.913628 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:53:12.914114 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:12.914072 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:53:12.914703 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:12.914685 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:53:14.368410 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:14.368364 2563 patch_prober.go:28] interesting pod/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 06:53:14.368803 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:14.368422 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" podUID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 06:53:16.453141 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:16.453107 2563 generic.go:356] "Generic (PLEG): container finished" podID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerID="460a6b776731577383445480b0280f4923f6aae3300742af113cb689f6f86f33" exitCode=0 Jul 21 06:53:16.453585 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:16.453179 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" event={"ID":"69ceafc8-bc72-4e90-8ebb-bf4feedffebf","Type":"ContainerDied","Data":"460a6b776731577383445480b0280f4923f6aae3300742af113cb689f6f86f33"} Jul 21 06:53:16.453627 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:16.453596 2563 scope.go:117] "RemoveContainer" containerID="460a6b776731577383445480b0280f4923f6aae3300742af113cb689f6f86f33" Jul 21 06:53:16.454706 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:16.454683 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:53:16.454789 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:16.454744 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:53:16.454789 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:16.454776 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:53:17.465041 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:17.465005 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" event={"ID":"69ceafc8-bc72-4e90-8ebb-bf4feedffebf","Type":"ContainerStarted","Data":"5b78650445fddda0e2c29708c05b9f2ac7d74fc3680c596c355ffac8d794abdb"} Jul 21 06:53:17.754424 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:17.754319 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:53:17.754790 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:17.754763 2563 patch_prober.go:28] interesting pod/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 21 06:53:17.754976 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:17.754944 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" podUID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 21 06:53:18.754241 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:18.754184 2563 patch_prober.go:28] interesting pod/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 21 06:53:18.755237 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:18.754821 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" podUID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 21 06:53:19.754418 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:19.754380 2563 patch_prober.go:28] interesting pod/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 21 06:53:19.755117 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:19.755083 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" podUID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 21 06:53:20.755191 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:20.755154 2563 patch_prober.go:28] interesting pod/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 21 06:53:20.755800 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:20.755751 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" podUID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 21 06:53:21.758385 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:21.758353 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:53:21.758854 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:21.758733 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:53:21.759594 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:21.759577 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:53:50.777906 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:53:50.777831 2563 scope.go:117] "RemoveContainer" containerID="c9e49d7a40ac9ebc132dcf16368ae362bbacf88582dfbd17872d0824400d58c3" Jul 21 06:55:07.907270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:07.907233 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6"] Jul 21 06:55:07.907755 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:07.907435 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerName="istio-proxy" containerID="cri-o://348cfb5e47404ab364157fc0823755100577bc4608a20c4dff3aaed7aa622015" gracePeriod=30 Jul 21 06:55:12.918915 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:12.918838 2563 patch_prober.go:28] interesting pod/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:56462->10.134.0.63:15021: read: connection reset by peer" start-of-body= Jul 21 06:55:12.919292 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:12.918903 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:56462->10.134.0.63:15021: read: connection reset by peer" Jul 21 06:55:13.150412 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.150241 2563 generic.go:356] "Generic (PLEG): container finished" podID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerID="348cfb5e47404ab364157fc0823755100577bc4608a20c4dff3aaed7aa622015" exitCode=0 Jul 21 06:55:13.150412 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.150321 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" event={"ID":"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c","Type":"ContainerDied","Data":"348cfb5e47404ab364157fc0823755100577bc4608a20c4dff3aaed7aa622015"} Jul 21 06:55:13.150412 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.150388 2563 scope.go:117] "RemoveContainer" containerID="71e5ab8e0ad8a4a2f49c212f2536b26e82351a94d71f771328d39b620fb967ba" Jul 21 06:55:13.176399 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.176374 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:55:13.282271 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.282243 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2ldp\" (UniqueName: \"kubernetes.io/projected/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-kube-api-access-g2ldp\") pod \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " Jul 21 06:55:13.282457 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.282303 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-credential-socket\") pod \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " Jul 21 06:55:13.282457 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.282348 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-podinfo\") pod \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " Jul 21 06:55:13.282457 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.282366 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-workload-certs\") pod \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " Jul 21 06:55:13.282457 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.282425 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istiod-ca-cert\") pod \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " Jul 21 06:55:13.282674 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.282465 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-data\") pod \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " Jul 21 06:55:13.282674 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.282512 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-workload-socket\") pod \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " Jul 21 06:55:13.282674 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.282541 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-envoy\") pod \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " Jul 21 06:55:13.282674 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.282581 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-token\") pod \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\" (UID: \"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c\") " Jul 21 06:55:13.282880 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.282720 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" (UID: "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:55:13.282880 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.282752 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" (UID: "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:55:13.283000 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.282878 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" (UID: "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:55:13.283000 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.282944 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" (UID: "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:55:13.283185 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.283060 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-workload-certs\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:13.283185 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.283182 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istiod-ca-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:13.283424 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.283196 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-workload-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:13.283424 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.283253 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-data" (OuterVolumeSpecName: "istio-data") pod "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" (UID: "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:55:13.284631 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.284611 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" (UID: "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 06:55:13.284725 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.284655 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-kube-api-access-g2ldp" (OuterVolumeSpecName: "kube-api-access-g2ldp") pod "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" (UID: "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c"). InnerVolumeSpecName "kube-api-access-g2ldp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:55:13.285381 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.285356 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" (UID: "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:55:13.285464 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.285419 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-token" (OuterVolumeSpecName: "istio-token") pod "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" (UID: "a7ce8023-f33e-4dd5-a9ab-4b317a958f4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:55:13.384512 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.384489 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-envoy\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:13.384512 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.384510 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-token\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:13.384651 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.384519 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2ldp\" (UniqueName: \"kubernetes.io/projected/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-kube-api-access-g2ldp\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:13.384651 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.384527 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-credential-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:13.384651 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.384538 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-podinfo\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:13.384651 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:13.384546 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c-istio-data\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:14.157574 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:14.157539 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" Jul 21 06:55:14.157574 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:14.157572 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6" event={"ID":"a7ce8023-f33e-4dd5-a9ab-4b317a958f4c","Type":"ContainerDied","Data":"50e695873e4d082f88b4c39838275604d7e74f96b50c813e07c2558877e01b3c"} Jul 21 06:55:14.158078 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:14.157617 2563 scope.go:117] "RemoveContainer" containerID="348cfb5e47404ab364157fc0823755100577bc4608a20c4dff3aaed7aa622015" Jul 21 06:55:14.197843 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:14.197807 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6"] Jul 21 06:55:14.202038 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:14.202008 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6d945c-openshift-default-6464ff88f5-47xc6"] Jul 21 06:55:14.486866 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:14.486787 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" path="/var/lib/kubelet/pods/a7ce8023-f33e-4dd5-a9ab-4b317a958f4c/volumes" Jul 21 06:55:30.266686 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:30.266652 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt"] Jul 21 06:55:30.267291 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:30.266888 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" podUID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerName="istio-proxy" containerID="cri-o://5b78650445fddda0e2c29708c05b9f2ac7d74fc3680c596c355ffac8d794abdb" gracePeriod=30 Jul 21 06:55:31.231891 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.231855 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7"] Jul 21 06:55:31.232294 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.232279 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerName="istio-proxy" Jul 21 06:55:31.232294 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.232295 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerName="istio-proxy" Jul 21 06:55:31.232399 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.232312 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerName="istio-proxy" Jul 21 06:55:31.232399 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.232317 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerName="istio-proxy" Jul 21 06:55:31.232399 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.232397 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerName="istio-proxy" Jul 21 06:55:31.232399 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.232406 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7ce8023-f33e-4dd5-a9ab-4b317a958f4c" containerName="istio-proxy" Jul 21 06:55:31.235408 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.235383 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.241548 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.241520 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-915db7-openshift-default-dockercfg-tv9ln\"" Jul 21 06:55:31.251582 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.251559 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-istio-envoy\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.251685 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.251622 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-workload-certs\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.251731 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.251687 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng87k\" (UniqueName: \"kubernetes.io/projected/d4cb755b-075e-408c-a807-befea00ebada-kube-api-access-ng87k\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.251770 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.251740 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4cb755b-075e-408c-a807-befea00ebada-istiod-ca-cert\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.251833 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.251814 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-istio-data\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.251870 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.251851 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-workload-socket\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.251925 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.251872 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-credential-socket\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.251925 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.251889 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4cb755b-075e-408c-a807-befea00ebada-istio-podinfo\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.251925 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.251906 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4cb755b-075e-408c-a807-befea00ebada-istio-token\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.281948 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.281923 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7"] Jul 21 06:55:31.352968 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.352929 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng87k\" (UniqueName: \"kubernetes.io/projected/d4cb755b-075e-408c-a807-befea00ebada-kube-api-access-ng87k\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.353380 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.353357 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4cb755b-075e-408c-a807-befea00ebada-istiod-ca-cert\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.353600 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.353581 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-istio-data\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.353724 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.353710 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-workload-socket\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.353825 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.353812 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-credential-socket\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.353955 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.353920 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4cb755b-075e-408c-a807-befea00ebada-istio-podinfo\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.353955 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.353951 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4cb755b-075e-408c-a807-befea00ebada-istio-token\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.354164 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.354004 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-istio-envoy\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.354164 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.354119 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-workload-certs\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.354358 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.354176 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4cb755b-075e-408c-a807-befea00ebada-istiod-ca-cert\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.354533 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.354505 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-workload-certs\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.354533 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.354522 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-credential-socket\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.355296 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.354771 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-istio-data\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.355296 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.355002 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-workload-socket\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.357637 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.357614 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-istio-envoy\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.357757 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.357736 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4cb755b-075e-408c-a807-befea00ebada-istio-podinfo\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.364633 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.364602 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng87k\" (UniqueName: \"kubernetes.io/projected/d4cb755b-075e-408c-a807-befea00ebada-kube-api-access-ng87k\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.364850 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.364834 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4cb755b-075e-408c-a807-befea00ebada-istio-token\") pod \"e2e-api-a-915db7-openshift-default-85488df7f5-crbg7\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.547269 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.547162 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:31.691047 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.691008 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7"] Jul 21 06:55:31.691959 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:55:31.691935 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4cb755b_075e_408c_a807_befea00ebada.slice/crio-0e09140412c5a759fd47a755a54de307b65063bd8e28463f820797055e2c5a7f WatchSource:0}: Error finding container 0e09140412c5a759fd47a755a54de307b65063bd8e28463f820797055e2c5a7f: Status 404 returned error can't find the container with id 0e09140412c5a759fd47a755a54de307b65063bd8e28463f820797055e2c5a7f Jul 21 06:55:31.694587 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.694557 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:55:31.694665 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.694619 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:55:31.694665 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:31.694644 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:55:32.272673 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:32.272638 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" event={"ID":"d4cb755b-075e-408c-a807-befea00ebada","Type":"ContainerStarted","Data":"0904b59122dc5cb353a3a66c4f67abc412ae4fc63a985ddd955eb8c5be985f21"} Jul 21 06:55:32.272855 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:32.272716 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" event={"ID":"d4cb755b-075e-408c-a807-befea00ebada","Type":"ContainerStarted","Data":"0e09140412c5a759fd47a755a54de307b65063bd8e28463f820797055e2c5a7f"} Jul 21 06:55:32.298405 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:32.298360 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" podStartSLOduration=1.29834487 podStartE2EDuration="1.29834487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:55:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:55:32.295798564 +0000 UTC m=+2262.241437221" watchObservedRunningTime="2026-07-21 06:55:32.29834487 +0000 UTC m=+2262.243983527" Jul 21 06:55:32.547703 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:32.547623 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:32.552817 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:32.552789 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:33.278800 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:33.278769 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:33.279603 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:33.279585 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:35.535092 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.534999 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:55:35.598826 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.598800 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-data\") pod \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " Jul 21 06:55:35.598949 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.598884 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqzqf\" (UniqueName: \"kubernetes.io/projected/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-kube-api-access-tqzqf\") pod \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " Jul 21 06:55:35.598949 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.598917 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-credential-socket\") pod \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " Jul 21 06:55:35.598949 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.598944 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istiod-ca-cert\") pod \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " Jul 21 06:55:35.599061 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.599040 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-token\") pod \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " Jul 21 06:55:35.599102 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.599080 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-workload-socket\") pod \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " Jul 21 06:55:35.599141 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.599107 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-envoy\") pod \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " Jul 21 06:55:35.599141 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.599132 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-podinfo\") pod \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " Jul 21 06:55:35.599281 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.599166 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-workload-certs\") pod \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\" (UID: \"69ceafc8-bc72-4e90-8ebb-bf4feedffebf\") " Jul 21 06:55:35.599281 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.599240 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "69ceafc8-bc72-4e90-8ebb-bf4feedffebf" (UID: "69ceafc8-bc72-4e90-8ebb-bf4feedffebf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:55:35.599374 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.599340 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "69ceafc8-bc72-4e90-8ebb-bf4feedffebf" (UID: "69ceafc8-bc72-4e90-8ebb-bf4feedffebf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:55:35.599432 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.599374 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "69ceafc8-bc72-4e90-8ebb-bf4feedffebf" (UID: "69ceafc8-bc72-4e90-8ebb-bf4feedffebf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:55:35.599574 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.599510 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-workload-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:35.599574 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.599535 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-credential-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:35.599574 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.599550 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istiod-ca-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:35.599574 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.599549 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "69ceafc8-bc72-4e90-8ebb-bf4feedffebf" (UID: "69ceafc8-bc72-4e90-8ebb-bf4feedffebf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:55:35.599871 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.599674 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-data" (OuterVolumeSpecName: "istio-data") pod "69ceafc8-bc72-4e90-8ebb-bf4feedffebf" (UID: "69ceafc8-bc72-4e90-8ebb-bf4feedffebf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:55:35.601499 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.601452 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-token" (OuterVolumeSpecName: "istio-token") pod "69ceafc8-bc72-4e90-8ebb-bf4feedffebf" (UID: "69ceafc8-bc72-4e90-8ebb-bf4feedffebf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:55:35.601499 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.601486 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "69ceafc8-bc72-4e90-8ebb-bf4feedffebf" (UID: "69ceafc8-bc72-4e90-8ebb-bf4feedffebf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 06:55:35.601667 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.601503 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-kube-api-access-tqzqf" (OuterVolumeSpecName: "kube-api-access-tqzqf") pod "69ceafc8-bc72-4e90-8ebb-bf4feedffebf" (UID: "69ceafc8-bc72-4e90-8ebb-bf4feedffebf"). InnerVolumeSpecName "kube-api-access-tqzqf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:55:35.601667 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.601522 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "69ceafc8-bc72-4e90-8ebb-bf4feedffebf" (UID: "69ceafc8-bc72-4e90-8ebb-bf4feedffebf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:55:35.700935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.700901 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-token\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:35.700935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.700929 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-envoy\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:35.700935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.700939 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-podinfo\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:35.700935 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.700947 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-workload-certs\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:35.701191 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.700956 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-istio-data\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:35.701191 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:35.700963 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqzqf\" (UniqueName: \"kubernetes.io/projected/69ceafc8-bc72-4e90-8ebb-bf4feedffebf-kube-api-access-tqzqf\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:55:36.296997 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:36.296967 2563 generic.go:356] "Generic (PLEG): container finished" podID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerID="5b78650445fddda0e2c29708c05b9f2ac7d74fc3680c596c355ffac8d794abdb" exitCode=0 Jul 21 06:55:36.297193 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:36.297016 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" Jul 21 06:55:36.297193 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:36.297062 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" event={"ID":"69ceafc8-bc72-4e90-8ebb-bf4feedffebf","Type":"ContainerDied","Data":"5b78650445fddda0e2c29708c05b9f2ac7d74fc3680c596c355ffac8d794abdb"} Jul 21 06:55:36.297193 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:36.297126 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt" event={"ID":"69ceafc8-bc72-4e90-8ebb-bf4feedffebf","Type":"ContainerDied","Data":"2dc5cf8bcdb68add85efd4c5e4c334691eca7174f4c71c3a17434f40c6b9877c"} Jul 21 06:55:36.297193 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:36.297156 2563 scope.go:117] "RemoveContainer" containerID="5b78650445fddda0e2c29708c05b9f2ac7d74fc3680c596c355ffac8d794abdb" Jul 21 06:55:36.307539 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:36.307516 2563 scope.go:117] "RemoveContainer" containerID="460a6b776731577383445480b0280f4923f6aae3300742af113cb689f6f86f33" Jul 21 06:55:36.316234 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:36.316196 2563 scope.go:117] "RemoveContainer" containerID="5b78650445fddda0e2c29708c05b9f2ac7d74fc3680c596c355ffac8d794abdb" Jul 21 06:55:36.316464 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:55:36.316446 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b78650445fddda0e2c29708c05b9f2ac7d74fc3680c596c355ffac8d794abdb\": container with ID starting with 5b78650445fddda0e2c29708c05b9f2ac7d74fc3680c596c355ffac8d794abdb not found: ID does not exist" containerID="5b78650445fddda0e2c29708c05b9f2ac7d74fc3680c596c355ffac8d794abdb" Jul 21 06:55:36.316516 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:36.316474 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b78650445fddda0e2c29708c05b9f2ac7d74fc3680c596c355ffac8d794abdb"} err="failed to get container status \"5b78650445fddda0e2c29708c05b9f2ac7d74fc3680c596c355ffac8d794abdb\": rpc error: code = NotFound desc = could not find container \"5b78650445fddda0e2c29708c05b9f2ac7d74fc3680c596c355ffac8d794abdb\": container with ID starting with 5b78650445fddda0e2c29708c05b9f2ac7d74fc3680c596c355ffac8d794abdb not found: ID does not exist" Jul 21 06:55:36.316516 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:36.316491 2563 scope.go:117] "RemoveContainer" containerID="460a6b776731577383445480b0280f4923f6aae3300742af113cb689f6f86f33" Jul 21 06:55:36.316734 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:55:36.316717 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"460a6b776731577383445480b0280f4923f6aae3300742af113cb689f6f86f33\": container with ID starting with 460a6b776731577383445480b0280f4923f6aae3300742af113cb689f6f86f33 not found: ID does not exist" containerID="460a6b776731577383445480b0280f4923f6aae3300742af113cb689f6f86f33" Jul 21 06:55:36.316792 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:36.316743 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"460a6b776731577383445480b0280f4923f6aae3300742af113cb689f6f86f33"} err="failed to get container status \"460a6b776731577383445480b0280f4923f6aae3300742af113cb689f6f86f33\": rpc error: code = NotFound desc = could not find container \"460a6b776731577383445480b0280f4923f6aae3300742af113cb689f6f86f33\": container with ID starting with 460a6b776731577383445480b0280f4923f6aae3300742af113cb689f6f86f33 not found: ID does not exist" Jul 21 06:55:36.321564 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:36.321543 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt"] Jul 21 06:55:36.328007 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:36.327987 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-11b68c-openshift-default-79f8db89f7-m5mrt"] Jul 21 06:55:36.487683 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:36.487648 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" path="/var/lib/kubelet/pods/69ceafc8-bc72-4e90-8ebb-bf4feedffebf/volumes" Jul 21 06:55:39.628627 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.628596 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr"] Jul 21 06:55:39.629046 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.629033 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerName="istio-proxy" Jul 21 06:55:39.629086 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.629048 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerName="istio-proxy" Jul 21 06:55:39.629086 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.629056 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerName="istio-proxy" Jul 21 06:55:39.629086 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.629061 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerName="istio-proxy" Jul 21 06:55:39.629182 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.629118 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerName="istio-proxy" Jul 21 06:55:39.629182 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.629127 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ceafc8-bc72-4e90-8ebb-bf4feedffebf" containerName="istio-proxy" Jul 21 06:55:39.633638 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.633623 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.637361 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.637310 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-c77bc7-openshift-default-dockercfg-5n6l8\"" Jul 21 06:55:39.649267 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.649245 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr"] Jul 21 06:55:39.743845 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.743807 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-credential-socket\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.744059 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.744041 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-podinfo\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.744175 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.744162 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxvmd\" (UniqueName: \"kubernetes.io/projected/af00e75b-f285-4633-95eb-cb99d8012a8d-kube-api-access-sxvmd\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.744327 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.744302 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-data\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.744459 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.744407 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af00e75b-f285-4633-95eb-cb99d8012a8d-istiod-ca-cert\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.744459 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.744453 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-workload-socket\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.744595 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.744487 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-workload-certs\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.744595 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.744561 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-token\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.744709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.744597 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-envoy\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.846277 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.846239 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-token\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.846441 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.846302 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-envoy\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.846441 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.846366 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-credential-socket\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.846441 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.846392 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-podinfo\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.846441 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.846418 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxvmd\" (UniqueName: \"kubernetes.io/projected/af00e75b-f285-4633-95eb-cb99d8012a8d-kube-api-access-sxvmd\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.846626 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.846600 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-data\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.846730 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.846697 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af00e75b-f285-4633-95eb-cb99d8012a8d-istiod-ca-cert\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.846856 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.846762 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-workload-socket\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.846856 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.846797 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-workload-certs\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.846856 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.846829 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-credential-socket\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.847095 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.847062 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-workload-socket\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.847336 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.847314 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-data\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.847414 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.847343 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-workload-certs\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.847469 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.847436 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af00e75b-f285-4633-95eb-cb99d8012a8d-istiod-ca-cert\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.848739 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.848721 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-envoy\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.848836 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.848787 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-podinfo\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.863130 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.863085 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxvmd\" (UniqueName: \"kubernetes.io/projected/af00e75b-f285-4633-95eb-cb99d8012a8d-kube-api-access-sxvmd\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.863940 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.863909 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-token\") pod \"e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:39.948077 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:39.948009 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:40.100046 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:40.100022 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr"] Jul 21 06:55:40.102227 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:55:40.102189 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf00e75b_f285_4633_95eb_cb99d8012a8d.slice/crio-70f4062b96eb6f29b5e46c4eb843c9924166ba620c2ab28839fd487dfa1dc655 WatchSource:0}: Error finding container 70f4062b96eb6f29b5e46c4eb843c9924166ba620c2ab28839fd487dfa1dc655: Status 404 returned error can't find the container with id 70f4062b96eb6f29b5e46c4eb843c9924166ba620c2ab28839fd487dfa1dc655 Jul 21 06:55:40.104338 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:40.104308 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:55:40.104412 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:40.104374 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:55:40.104412 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:40.104401 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:55:40.323467 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:40.323438 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" event={"ID":"af00e75b-f285-4633-95eb-cb99d8012a8d","Type":"ContainerStarted","Data":"b3cdffd4e7d97ff22760f9bfdc46bfac9052b2221ca20e9cc9966f0fd87639fd"} Jul 21 06:55:40.323646 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:40.323497 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" event={"ID":"af00e75b-f285-4633-95eb-cb99d8012a8d","Type":"ContainerStarted","Data":"70f4062b96eb6f29b5e46c4eb843c9924166ba620c2ab28839fd487dfa1dc655"} Jul 21 06:55:40.365899 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:40.365838 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" podStartSLOduration=1.365820449 podStartE2EDuration="1.365820449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:55:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:55:40.362308783 +0000 UTC m=+2270.307947441" watchObservedRunningTime="2026-07-21 06:55:40.365820449 +0000 UTC m=+2270.311459105" Jul 21 06:55:40.948438 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:40.948406 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:41.953962 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:41.953916 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:42.335328 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:42.335297 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:42.336329 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:42.336307 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:55:49.235798 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:49.235276 2563 patch_prober.go:28] interesting pod/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": read tcp 10.134.0.2:48080->10.134.0.65:15021: read: connection reset by peer" start-of-body= Jul 21 06:55:49.235798 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:49.235354 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" podUID="d4cb755b-075e-408c-a807-befea00ebada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": read tcp 10.134.0.2:48080->10.134.0.65:15021: read: connection reset by peer" Jul 21 06:55:49.394797 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:49.394762 2563 generic.go:356] "Generic (PLEG): container finished" podID="d4cb755b-075e-408c-a807-befea00ebada" containerID="0904b59122dc5cb353a3a66c4f67abc412ae4fc63a985ddd955eb8c5be985f21" exitCode=0 Jul 21 06:55:49.394975 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:49.394831 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" event={"ID":"d4cb755b-075e-408c-a807-befea00ebada","Type":"ContainerDied","Data":"0904b59122dc5cb353a3a66c4f67abc412ae4fc63a985ddd955eb8c5be985f21"} Jul 21 06:55:49.395270 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:49.395254 2563 scope.go:117] "RemoveContainer" containerID="0904b59122dc5cb353a3a66c4f67abc412ae4fc63a985ddd955eb8c5be985f21" Jul 21 06:55:49.396317 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:49.396291 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:55:49.396424 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:49.396373 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:55:49.396493 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:49.396422 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:55:50.401566 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:50.401526 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" event={"ID":"d4cb755b-075e-408c-a807-befea00ebada","Type":"ContainerStarted","Data":"57fda6215f1a1e5a300ff02ac2fa8ccfc183e6e4df861ca0126bb6eae4d6b132"} Jul 21 06:55:50.547805 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:50.547704 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:50.548010 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:50.547988 2563 patch_prober.go:28] interesting pod/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Jul 21 06:55:50.548085 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:50.548061 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" podUID="d4cb755b-075e-408c-a807-befea00ebada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Jul 21 06:55:50.808468 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:50.808387 2563 scope.go:117] "RemoveContainer" containerID="1c3d3fae586fd97a42830f8403230b99a3c6394835a8effc38309dced6aa716c" Jul 21 06:55:51.548449 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:51.548411 2563 patch_prober.go:28] interesting pod/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Jul 21 06:55:51.549508 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:51.549021 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" podUID="d4cb755b-075e-408c-a807-befea00ebada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Jul 21 06:55:52.548584 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:52.548542 2563 patch_prober.go:28] interesting pod/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Jul 21 06:55:52.549081 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:52.548626 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" podUID="d4cb755b-075e-408c-a807-befea00ebada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Jul 21 06:55:53.548269 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:53.548210 2563 patch_prober.go:28] interesting pod/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Jul 21 06:55:53.548478 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:53.548313 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" podUID="d4cb755b-075e-408c-a807-befea00ebada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Jul 21 06:55:54.556726 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:54.556689 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:54.558019 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:54.557966 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:54.559118 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:54.559072 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:55:58.378390 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:58.378350 2563 patch_prober.go:28] interesting pod/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 06:55:58.378822 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:58.378425 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" podUID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 06:55:59.460848 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:59.460811 2563 generic.go:356] "Generic (PLEG): container finished" podID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerID="b3cdffd4e7d97ff22760f9bfdc46bfac9052b2221ca20e9cc9966f0fd87639fd" exitCode=0 Jul 21 06:55:59.461392 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:59.460873 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" event={"ID":"af00e75b-f285-4633-95eb-cb99d8012a8d","Type":"ContainerDied","Data":"b3cdffd4e7d97ff22760f9bfdc46bfac9052b2221ca20e9cc9966f0fd87639fd"} Jul 21 06:55:59.461392 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:59.461378 2563 scope.go:117] "RemoveContainer" containerID="b3cdffd4e7d97ff22760f9bfdc46bfac9052b2221ca20e9cc9966f0fd87639fd" Jul 21 06:55:59.462502 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:59.462474 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:55:59.462573 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:59.462547 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:55:59.462614 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:55:59.462578 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:56:00.471287 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:00.471254 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" event={"ID":"af00e75b-f285-4633-95eb-cb99d8012a8d","Type":"ContainerStarted","Data":"ad06e1b8e2276c9c62af7dd9a6f0d4de6c0210d927d4a0874bdbae867a2f90c5"} Jul 21 06:56:00.949285 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:00.948431 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:56:00.949285 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:00.948912 2563 patch_prober.go:28] interesting pod/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 21 06:56:00.949285 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:00.948968 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" podUID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 21 06:56:01.948557 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:01.948518 2563 patch_prober.go:28] interesting pod/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 21 06:56:01.949326 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:01.949200 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" podUID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 21 06:56:02.949150 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:02.949110 2563 patch_prober.go:28] interesting pod/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 21 06:56:02.949748 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:02.949716 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" podUID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 21 06:56:03.948456 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:03.948421 2563 patch_prober.go:28] interesting pod/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 21 06:56:03.948753 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:03.948725 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" podUID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 21 06:56:04.952560 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:04.952530 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:56:04.952947 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:04.952923 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:56:04.953718 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:04.953692 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:56:22.849230 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:22.849192 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7"] Jul 21 06:56:22.849624 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:22.849454 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" podUID="d4cb755b-075e-408c-a807-befea00ebada" containerName="istio-proxy" containerID="cri-o://57fda6215f1a1e5a300ff02ac2fa8ccfc183e6e4df861ca0126bb6eae4d6b132" gracePeriod=30 Jul 21 06:56:24.560750 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:24.560718 2563 patch_prober.go:28] interesting pod/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 06:56:24.561134 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:24.560783 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" podUID="d4cb755b-075e-408c-a807-befea00ebada" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 06:56:28.103577 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.103552 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:56:28.247882 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.247797 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-credential-socket\") pod \"d4cb755b-075e-408c-a807-befea00ebada\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " Jul 21 06:56:28.247882 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.247841 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-istio-data\") pod \"d4cb755b-075e-408c-a807-befea00ebada\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " Jul 21 06:56:28.247882 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.247864 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-istio-envoy\") pod \"d4cb755b-075e-408c-a807-befea00ebada\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " Jul 21 06:56:28.247882 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.247885 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4cb755b-075e-408c-a807-befea00ebada-istiod-ca-cert\") pod \"d4cb755b-075e-408c-a807-befea00ebada\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " Jul 21 06:56:28.248248 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.247924 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-workload-socket\") pod \"d4cb755b-075e-408c-a807-befea00ebada\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " Jul 21 06:56:28.248248 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.247968 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4cb755b-075e-408c-a807-befea00ebada-istio-podinfo\") pod \"d4cb755b-075e-408c-a807-befea00ebada\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " Jul 21 06:56:28.248248 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.248108 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ng87k\" (UniqueName: \"kubernetes.io/projected/d4cb755b-075e-408c-a807-befea00ebada-kube-api-access-ng87k\") pod \"d4cb755b-075e-408c-a807-befea00ebada\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " Jul 21 06:56:28.248248 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.248146 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d4cb755b-075e-408c-a807-befea00ebada" (UID: "d4cb755b-075e-408c-a807-befea00ebada"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:56:28.248248 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.248159 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-workload-certs\") pod \"d4cb755b-075e-408c-a807-befea00ebada\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " Jul 21 06:56:28.248513 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.248288 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4cb755b-075e-408c-a807-befea00ebada-istio-token\") pod \"d4cb755b-075e-408c-a807-befea00ebada\" (UID: \"d4cb755b-075e-408c-a807-befea00ebada\") " Jul 21 06:56:28.248513 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.248325 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d4cb755b-075e-408c-a807-befea00ebada" (UID: "d4cb755b-075e-408c-a807-befea00ebada"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:56:28.248513 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.248401 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d4cb755b-075e-408c-a807-befea00ebada" (UID: "d4cb755b-075e-408c-a807-befea00ebada"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:56:28.248818 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.248639 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-credential-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:28.248818 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.248657 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-workload-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:28.248818 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.248670 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-workload-certs\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:28.248994 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.248853 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-istio-data" (OuterVolumeSpecName: "istio-data") pod "d4cb755b-075e-408c-a807-befea00ebada" (UID: "d4cb755b-075e-408c-a807-befea00ebada"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:56:28.248994 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.248836 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4cb755b-075e-408c-a807-befea00ebada-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d4cb755b-075e-408c-a807-befea00ebada" (UID: "d4cb755b-075e-408c-a807-befea00ebada"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:56:28.250382 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.250356 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d4cb755b-075e-408c-a807-befea00ebada" (UID: "d4cb755b-075e-408c-a807-befea00ebada"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:56:28.250382 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.250361 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4cb755b-075e-408c-a807-befea00ebada-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d4cb755b-075e-408c-a807-befea00ebada" (UID: "d4cb755b-075e-408c-a807-befea00ebada"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 06:56:28.250558 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.250391 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4cb755b-075e-408c-a807-befea00ebada-kube-api-access-ng87k" (OuterVolumeSpecName: "kube-api-access-ng87k") pod "d4cb755b-075e-408c-a807-befea00ebada" (UID: "d4cb755b-075e-408c-a807-befea00ebada"). InnerVolumeSpecName "kube-api-access-ng87k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:56:28.250621 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.250582 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4cb755b-075e-408c-a807-befea00ebada-istio-token" (OuterVolumeSpecName: "istio-token") pod "d4cb755b-075e-408c-a807-befea00ebada" (UID: "d4cb755b-075e-408c-a807-befea00ebada"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:56:28.349693 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.349667 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng87k\" (UniqueName: \"kubernetes.io/projected/d4cb755b-075e-408c-a807-befea00ebada-kube-api-access-ng87k\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:28.349693 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.349691 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4cb755b-075e-408c-a807-befea00ebada-istio-token\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:28.349846 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.349700 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-istio-data\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:28.349846 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.349708 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4cb755b-075e-408c-a807-befea00ebada-istio-envoy\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:28.349846 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.349716 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4cb755b-075e-408c-a807-befea00ebada-istiod-ca-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:28.349846 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.349723 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4cb755b-075e-408c-a807-befea00ebada-istio-podinfo\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:28.636823 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.636794 2563 generic.go:356] "Generic (PLEG): container finished" podID="d4cb755b-075e-408c-a807-befea00ebada" containerID="57fda6215f1a1e5a300ff02ac2fa8ccfc183e6e4df861ca0126bb6eae4d6b132" exitCode=0 Jul 21 06:56:28.636961 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.636856 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" Jul 21 06:56:28.636961 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.636861 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" event={"ID":"d4cb755b-075e-408c-a807-befea00ebada","Type":"ContainerDied","Data":"57fda6215f1a1e5a300ff02ac2fa8ccfc183e6e4df861ca0126bb6eae4d6b132"} Jul 21 06:56:28.636961 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.636901 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7" event={"ID":"d4cb755b-075e-408c-a807-befea00ebada","Type":"ContainerDied","Data":"0e09140412c5a759fd47a755a54de307b65063bd8e28463f820797055e2c5a7f"} Jul 21 06:56:28.636961 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.636921 2563 scope.go:117] "RemoveContainer" containerID="57fda6215f1a1e5a300ff02ac2fa8ccfc183e6e4df861ca0126bb6eae4d6b132" Jul 21 06:56:28.646357 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.646339 2563 scope.go:117] "RemoveContainer" containerID="0904b59122dc5cb353a3a66c4f67abc412ae4fc63a985ddd955eb8c5be985f21" Jul 21 06:56:28.654920 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.654902 2563 scope.go:117] "RemoveContainer" containerID="57fda6215f1a1e5a300ff02ac2fa8ccfc183e6e4df861ca0126bb6eae4d6b132" Jul 21 06:56:28.655179 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:56:28.655160 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57fda6215f1a1e5a300ff02ac2fa8ccfc183e6e4df861ca0126bb6eae4d6b132\": container with ID starting with 57fda6215f1a1e5a300ff02ac2fa8ccfc183e6e4df861ca0126bb6eae4d6b132 not found: ID does not exist" containerID="57fda6215f1a1e5a300ff02ac2fa8ccfc183e6e4df861ca0126bb6eae4d6b132" Jul 21 06:56:28.655369 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.655193 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57fda6215f1a1e5a300ff02ac2fa8ccfc183e6e4df861ca0126bb6eae4d6b132"} err="failed to get container status \"57fda6215f1a1e5a300ff02ac2fa8ccfc183e6e4df861ca0126bb6eae4d6b132\": rpc error: code = NotFound desc = could not find container \"57fda6215f1a1e5a300ff02ac2fa8ccfc183e6e4df861ca0126bb6eae4d6b132\": container with ID starting with 57fda6215f1a1e5a300ff02ac2fa8ccfc183e6e4df861ca0126bb6eae4d6b132 not found: ID does not exist" Jul 21 06:56:28.655369 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.655234 2563 scope.go:117] "RemoveContainer" containerID="0904b59122dc5cb353a3a66c4f67abc412ae4fc63a985ddd955eb8c5be985f21" Jul 21 06:56:28.655478 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:56:28.655460 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0904b59122dc5cb353a3a66c4f67abc412ae4fc63a985ddd955eb8c5be985f21\": container with ID starting with 0904b59122dc5cb353a3a66c4f67abc412ae4fc63a985ddd955eb8c5be985f21 not found: ID does not exist" containerID="0904b59122dc5cb353a3a66c4f67abc412ae4fc63a985ddd955eb8c5be985f21" Jul 21 06:56:28.655515 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.655486 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0904b59122dc5cb353a3a66c4f67abc412ae4fc63a985ddd955eb8c5be985f21"} err="failed to get container status \"0904b59122dc5cb353a3a66c4f67abc412ae4fc63a985ddd955eb8c5be985f21\": rpc error: code = NotFound desc = could not find container \"0904b59122dc5cb353a3a66c4f67abc412ae4fc63a985ddd955eb8c5be985f21\": container with ID starting with 0904b59122dc5cb353a3a66c4f67abc412ae4fc63a985ddd955eb8c5be985f21 not found: ID does not exist" Jul 21 06:56:28.659840 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.659821 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7"] Jul 21 06:56:28.667887 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:28.667866 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-915db7-openshift-default-85488df7f5-crbg7"] Jul 21 06:56:30.487144 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:30.487116 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4cb755b-075e-408c-a807-befea00ebada" path="/var/lib/kubelet/pods/d4cb755b-075e-408c-a807-befea00ebada/volumes" Jul 21 06:56:50.839711 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:50.839627 2563 scope.go:117] "RemoveContainer" containerID="c4e0fb3df36de3d47c7a0af5821738f2dc85fd2c45b3f9103fe0c59971961f03" Jul 21 06:56:50.894879 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:50.894847 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr"] Jul 21 06:56:50.895078 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:50.895056 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" podUID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerName="istio-proxy" containerID="cri-o://ad06e1b8e2276c9c62af7dd9a6f0d4de6c0210d927d4a0874bdbae867a2f90c5" gracePeriod=30 Jul 21 06:56:51.921896 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.921860 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk"] Jul 21 06:56:51.923435 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.923404 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4cb755b-075e-408c-a807-befea00ebada" containerName="istio-proxy" Jul 21 06:56:51.923435 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.923434 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4cb755b-075e-408c-a807-befea00ebada" containerName="istio-proxy" Jul 21 06:56:51.923619 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.923453 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4cb755b-075e-408c-a807-befea00ebada" containerName="istio-proxy" Jul 21 06:56:51.923619 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.923462 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4cb755b-075e-408c-a807-befea00ebada" containerName="istio-proxy" Jul 21 06:56:51.923711 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.923674 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4cb755b-075e-408c-a807-befea00ebada" containerName="istio-proxy" Jul 21 06:56:51.923814 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.923722 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4cb755b-075e-408c-a807-befea00ebada" containerName="istio-proxy" Jul 21 06:56:51.928707 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.928689 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:51.933666 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.933646 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-08eb3e-openshift-default-dockercfg-n8vfl\"" Jul 21 06:56:51.944576 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.944549 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk"] Jul 21 06:56:51.969552 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.969530 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-envoy\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:51.969672 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.969570 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istiod-ca-cert\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:51.969672 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.969591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-data\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:51.969672 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.969606 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-podinfo\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:51.969823 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.969710 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-workload-certs\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:51.969823 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.969761 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs7x5\" (UniqueName: \"kubernetes.io/projected/87550db3-1a9e-4b21-a7ae-436618bb9d9c-kube-api-access-bs7x5\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:51.969933 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.969858 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-token\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:51.969933 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.969896 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-credential-socket\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:51.970034 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:51.969966 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-workload-socket\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.071150 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.071110 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-token\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.071368 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.071156 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-credential-socket\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.071368 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.071192 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-workload-socket\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.071368 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.071255 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-envoy\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.071368 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.071302 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istiod-ca-cert\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.071368 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.071326 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-data\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.071368 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.071346 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-podinfo\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.071682 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.071425 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-workload-certs\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.071682 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.071497 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs7x5\" (UniqueName: \"kubernetes.io/projected/87550db3-1a9e-4b21-a7ae-436618bb9d9c-kube-api-access-bs7x5\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.071682 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.071595 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-credential-socket\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.072068 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.072044 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-workload-certs\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.072239 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.072103 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istiod-ca-cert\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.072365 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.072238 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-data\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.072476 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.072326 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-workload-socket\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.079235 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.079191 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-envoy\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.079534 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.079513 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-podinfo\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.085470 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.085441 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-token\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.088825 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.088792 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs7x5\" (UniqueName: \"kubernetes.io/projected/87550db3-1a9e-4b21-a7ae-436618bb9d9c-kube-api-access-bs7x5\") pod \"e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.240145 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.240059 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:52.406442 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.406417 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk"] Jul 21 06:56:52.407654 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:56:52.407632 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87550db3_1a9e_4b21_a7ae_436618bb9d9c.slice/crio-0fc6b19efbd32b5c4e32b9573934677127897604f2c6b15139f041d894c7b2bf WatchSource:0}: Error finding container 0fc6b19efbd32b5c4e32b9573934677127897604f2c6b15139f041d894c7b2bf: Status 404 returned error can't find the container with id 0fc6b19efbd32b5c4e32b9573934677127897604f2c6b15139f041d894c7b2bf Jul 21 06:56:52.409843 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.409811 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:56:52.409936 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.409877 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:56:52.409936 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.409906 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:56:52.776404 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.776374 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" event={"ID":"87550db3-1a9e-4b21-a7ae-436618bb9d9c","Type":"ContainerStarted","Data":"bca0401738c2b155bfcbf284f8df7b9bc4b518c0fda2293567e395e80b7ec52a"} Jul 21 06:56:52.776602 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:52.776423 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" event={"ID":"87550db3-1a9e-4b21-a7ae-436618bb9d9c","Type":"ContainerStarted","Data":"0fc6b19efbd32b5c4e32b9573934677127897604f2c6b15139f041d894c7b2bf"} Jul 21 06:56:53.240417 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:53.240380 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:53.246518 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:53.246494 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:53.268844 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:53.268795 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" podStartSLOduration=2.268781041 podStartE2EDuration="2.268781041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:56:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:56:52.807493551 +0000 UTC m=+2342.753132209" watchObservedRunningTime="2026-07-21 06:56:53.268781041 +0000 UTC m=+2343.214419698" Jul 21 06:56:53.781730 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:53.781704 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:53.782729 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:53.782714 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:56:56.165189 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.165126 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:56:56.208095 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208065 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-token\") pod \"af00e75b-f285-4633-95eb-cb99d8012a8d\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " Jul 21 06:56:56.208280 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208118 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-podinfo\") pod \"af00e75b-f285-4633-95eb-cb99d8012a8d\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " Jul 21 06:56:56.208280 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208188 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-workload-certs\") pod \"af00e75b-f285-4633-95eb-cb99d8012a8d\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " Jul 21 06:56:56.208280 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208246 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-credential-socket\") pod \"af00e75b-f285-4633-95eb-cb99d8012a8d\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " Jul 21 06:56:56.208454 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208292 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxvmd\" (UniqueName: \"kubernetes.io/projected/af00e75b-f285-4633-95eb-cb99d8012a8d-kube-api-access-sxvmd\") pod \"af00e75b-f285-4633-95eb-cb99d8012a8d\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " Jul 21 06:56:56.208454 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208322 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-data\") pod \"af00e75b-f285-4633-95eb-cb99d8012a8d\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " Jul 21 06:56:56.208454 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208365 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af00e75b-f285-4633-95eb-cb99d8012a8d-istiod-ca-cert\") pod \"af00e75b-f285-4633-95eb-cb99d8012a8d\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " Jul 21 06:56:56.208454 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208418 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-workload-socket\") pod \"af00e75b-f285-4633-95eb-cb99d8012a8d\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " Jul 21 06:56:56.208666 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208455 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-envoy\") pod \"af00e75b-f285-4633-95eb-cb99d8012a8d\" (UID: \"af00e75b-f285-4633-95eb-cb99d8012a8d\") " Jul 21 06:56:56.208666 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208521 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "af00e75b-f285-4633-95eb-cb99d8012a8d" (UID: "af00e75b-f285-4633-95eb-cb99d8012a8d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:56:56.208666 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208555 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "af00e75b-f285-4633-95eb-cb99d8012a8d" (UID: "af00e75b-f285-4633-95eb-cb99d8012a8d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:56:56.208828 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208685 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "af00e75b-f285-4633-95eb-cb99d8012a8d" (UID: "af00e75b-f285-4633-95eb-cb99d8012a8d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:56:56.208828 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208819 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-workload-certs\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:56.208941 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208839 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-credential-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:56.208941 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.208895 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-workload-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:56.209076 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.209026 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af00e75b-f285-4633-95eb-cb99d8012a8d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "af00e75b-f285-4633-95eb-cb99d8012a8d" (UID: "af00e75b-f285-4633-95eb-cb99d8012a8d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 06:56:56.209599 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.209574 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-data" (OuterVolumeSpecName: "istio-data") pod "af00e75b-f285-4633-95eb-cb99d8012a8d" (UID: "af00e75b-f285-4633-95eb-cb99d8012a8d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:56:56.210655 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.210611 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "af00e75b-f285-4633-95eb-cb99d8012a8d" (UID: "af00e75b-f285-4633-95eb-cb99d8012a8d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 06:56:56.210772 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.210715 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "af00e75b-f285-4633-95eb-cb99d8012a8d" (UID: "af00e75b-f285-4633-95eb-cb99d8012a8d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 06:56:56.211010 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.210991 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-token" (OuterVolumeSpecName: "istio-token") pod "af00e75b-f285-4633-95eb-cb99d8012a8d" (UID: "af00e75b-f285-4633-95eb-cb99d8012a8d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:56:56.211081 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.211034 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af00e75b-f285-4633-95eb-cb99d8012a8d-kube-api-access-sxvmd" (OuterVolumeSpecName: "kube-api-access-sxvmd") pod "af00e75b-f285-4633-95eb-cb99d8012a8d" (UID: "af00e75b-f285-4633-95eb-cb99d8012a8d"). InnerVolumeSpecName "kube-api-access-sxvmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 06:56:56.309978 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.309943 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af00e75b-f285-4633-95eb-cb99d8012a8d-istiod-ca-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:56.309978 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.309972 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-envoy\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:56.309978 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.309984 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-token\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:56.310310 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.309995 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-podinfo\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:56.310310 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.310007 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxvmd\" (UniqueName: \"kubernetes.io/projected/af00e75b-f285-4633-95eb-cb99d8012a8d-kube-api-access-sxvmd\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:56.310310 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.310019 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af00e75b-f285-4633-95eb-cb99d8012a8d-istio-data\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 06:56:56.802767 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.802679 2563 generic.go:356] "Generic (PLEG): container finished" podID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerID="ad06e1b8e2276c9c62af7dd9a6f0d4de6c0210d927d4a0874bdbae867a2f90c5" exitCode=0 Jul 21 06:56:56.802767 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.802723 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" event={"ID":"af00e75b-f285-4633-95eb-cb99d8012a8d","Type":"ContainerDied","Data":"ad06e1b8e2276c9c62af7dd9a6f0d4de6c0210d927d4a0874bdbae867a2f90c5"} Jul 21 06:56:56.802767 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.802733 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" Jul 21 06:56:56.803005 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.802777 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr" event={"ID":"af00e75b-f285-4633-95eb-cb99d8012a8d","Type":"ContainerDied","Data":"70f4062b96eb6f29b5e46c4eb843c9924166ba620c2ab28839fd487dfa1dc655"} Jul 21 06:56:56.803005 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.802807 2563 scope.go:117] "RemoveContainer" containerID="ad06e1b8e2276c9c62af7dd9a6f0d4de6c0210d927d4a0874bdbae867a2f90c5" Jul 21 06:56:56.812521 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.812501 2563 scope.go:117] "RemoveContainer" containerID="b3cdffd4e7d97ff22760f9bfdc46bfac9052b2221ca20e9cc9966f0fd87639fd" Jul 21 06:56:56.821475 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.821454 2563 scope.go:117] "RemoveContainer" containerID="ad06e1b8e2276c9c62af7dd9a6f0d4de6c0210d927d4a0874bdbae867a2f90c5" Jul 21 06:56:56.821721 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:56:56.821703 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad06e1b8e2276c9c62af7dd9a6f0d4de6c0210d927d4a0874bdbae867a2f90c5\": container with ID starting with ad06e1b8e2276c9c62af7dd9a6f0d4de6c0210d927d4a0874bdbae867a2f90c5 not found: ID does not exist" containerID="ad06e1b8e2276c9c62af7dd9a6f0d4de6c0210d927d4a0874bdbae867a2f90c5" Jul 21 06:56:56.821805 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.821729 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad06e1b8e2276c9c62af7dd9a6f0d4de6c0210d927d4a0874bdbae867a2f90c5"} err="failed to get container status \"ad06e1b8e2276c9c62af7dd9a6f0d4de6c0210d927d4a0874bdbae867a2f90c5\": rpc error: code = NotFound desc = could not find container \"ad06e1b8e2276c9c62af7dd9a6f0d4de6c0210d927d4a0874bdbae867a2f90c5\": container with ID starting with ad06e1b8e2276c9c62af7dd9a6f0d4de6c0210d927d4a0874bdbae867a2f90c5 not found: ID does not exist" Jul 21 06:56:56.821805 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.821746 2563 scope.go:117] "RemoveContainer" containerID="b3cdffd4e7d97ff22760f9bfdc46bfac9052b2221ca20e9cc9966f0fd87639fd" Jul 21 06:56:56.821982 ip-10-0-140-159 kubenswrapper[2563]: E0721 06:56:56.821967 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3cdffd4e7d97ff22760f9bfdc46bfac9052b2221ca20e9cc9966f0fd87639fd\": container with ID starting with b3cdffd4e7d97ff22760f9bfdc46bfac9052b2221ca20e9cc9966f0fd87639fd not found: ID does not exist" containerID="b3cdffd4e7d97ff22760f9bfdc46bfac9052b2221ca20e9cc9966f0fd87639fd" Jul 21 06:56:56.822021 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.821986 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3cdffd4e7d97ff22760f9bfdc46bfac9052b2221ca20e9cc9966f0fd87639fd"} err="failed to get container status \"b3cdffd4e7d97ff22760f9bfdc46bfac9052b2221ca20e9cc9966f0fd87639fd\": rpc error: code = NotFound desc = could not find container \"b3cdffd4e7d97ff22760f9bfdc46bfac9052b2221ca20e9cc9966f0fd87639fd\": container with ID starting with b3cdffd4e7d97ff22760f9bfdc46bfac9052b2221ca20e9cc9966f0fd87639fd not found: ID does not exist" Jul 21 06:56:56.824011 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.823991 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr"] Jul 21 06:56:56.828769 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:56.828747 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-c77bc7-openshift-default-5f47b6f8c9-cgrhr"] Jul 21 06:56:58.488126 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:58.488090 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af00e75b-f285-4633-95eb-cb99d8012a8d" path="/var/lib/kubelet/pods/af00e75b-f285-4633-95eb-cb99d8012a8d/volumes" Jul 21 06:56:59.986375 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:59.986340 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk"] Jul 21 06:56:59.986997 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:59.986981 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerName="istio-proxy" Jul 21 06:56:59.987042 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:59.987000 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerName="istio-proxy" Jul 21 06:56:59.987042 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:59.987020 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerName="istio-proxy" Jul 21 06:56:59.987042 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:59.987029 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerName="istio-proxy" Jul 21 06:56:59.987139 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:59.987125 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerName="istio-proxy" Jul 21 06:56:59.987176 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:59.987143 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="af00e75b-f285-4633-95eb-cb99d8012a8d" containerName="istio-proxy" Jul 21 06:56:59.992297 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:59.992276 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:56:59.996009 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:56:59.995991 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-bddf64-openshift-default-dockercfg-mwt5g\"" Jul 21 06:57:00.007829 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.007808 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk"] Jul 21 06:57:00.045559 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.045525 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-workload-socket\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.045678 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.045572 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/353f759a-e896-4edc-8ec7-9ed803e20655-istiod-ca-cert\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.045678 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.045605 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkzgq\" (UniqueName: \"kubernetes.io/projected/353f759a-e896-4edc-8ec7-9ed803e20655-kube-api-access-kkzgq\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.045678 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.045669 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-istio-data\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.045834 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.045748 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-credential-socket\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.045834 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.045805 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-istio-envoy\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.046160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.045838 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/353f759a-e896-4edc-8ec7-9ed803e20655-istio-token\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.046160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.045889 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/353f759a-e896-4edc-8ec7-9ed803e20655-istio-podinfo\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.046160 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.045971 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-workload-certs\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.146687 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.146655 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-workload-certs\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.146852 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.146712 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-workload-socket\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.146852 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.146733 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/353f759a-e896-4edc-8ec7-9ed803e20655-istiod-ca-cert\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.146852 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.146751 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkzgq\" (UniqueName: \"kubernetes.io/projected/353f759a-e896-4edc-8ec7-9ed803e20655-kube-api-access-kkzgq\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.146852 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.146793 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-istio-data\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.146852 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.146815 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-credential-socket\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.146852 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.146836 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-istio-envoy\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.147201 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.146855 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/353f759a-e896-4edc-8ec7-9ed803e20655-istio-token\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.147201 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.146901 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/353f759a-e896-4edc-8ec7-9ed803e20655-istio-podinfo\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.147201 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.147100 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-workload-certs\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.147419 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.147199 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-workload-socket\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.147419 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.147277 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-istio-data\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.147419 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.147303 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-credential-socket\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.147587 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.147542 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/353f759a-e896-4edc-8ec7-9ed803e20655-istiod-ca-cert\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.149357 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.149324 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-istio-envoy\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.149576 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.149552 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/353f759a-e896-4edc-8ec7-9ed803e20655-istio-podinfo\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.156268 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.156243 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/353f759a-e896-4edc-8ec7-9ed803e20655-istio-token\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.156599 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.156577 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkzgq\" (UniqueName: \"kubernetes.io/projected/353f759a-e896-4edc-8ec7-9ed803e20655-kube-api-access-kkzgq\") pod \"e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.303548 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.303456 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:00.450554 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.450528 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk"] Jul 21 06:57:00.452552 ip-10-0-140-159 kubenswrapper[2563]: W0721 06:57:00.452522 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353f759a_e896_4edc_8ec7_9ed803e20655.slice/crio-bed8b1f20cb22f23b13b82f860b58735883fb0cb82cc6f6bfbcf89339a53ad54 WatchSource:0}: Error finding container bed8b1f20cb22f23b13b82f860b58735883fb0cb82cc6f6bfbcf89339a53ad54: Status 404 returned error can't find the container with id bed8b1f20cb22f23b13b82f860b58735883fb0cb82cc6f6bfbcf89339a53ad54 Jul 21 06:57:00.454862 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.454822 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:57:00.454949 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.454916 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:57:00.454987 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.454963 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:57:00.830312 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.830281 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" event={"ID":"353f759a-e896-4edc-8ec7-9ed803e20655","Type":"ContainerStarted","Data":"4a86e23af42cb42211942af5934289d85b8eafcbd35559da2e8788e0c5c60fd1"} Jul 21 06:57:00.830515 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.830340 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" event={"ID":"353f759a-e896-4edc-8ec7-9ed803e20655","Type":"ContainerStarted","Data":"bed8b1f20cb22f23b13b82f860b58735883fb0cb82cc6f6bfbcf89339a53ad54"} Jul 21 06:57:00.853567 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:00.853511 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" podStartSLOduration=1.853490536 podStartE2EDuration="1.853490536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 06:56:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 06:57:00.852010744 +0000 UTC m=+2350.797649422" watchObservedRunningTime="2026-07-21 06:57:00.853490536 +0000 UTC m=+2350.799129194" Jul 21 06:57:01.303796 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:01.303715 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:01.308293 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:01.308261 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:01.836428 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:01.836388 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:01.837379 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:01.837356 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:11.905673 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:11.905642 2563 generic.go:356] "Generic (PLEG): container finished" podID="87550db3-1a9e-4b21-a7ae-436618bb9d9c" containerID="bca0401738c2b155bfcbf284f8df7b9bc4b518c0fda2293567e395e80b7ec52a" exitCode=0 Jul 21 06:57:11.906101 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:11.905720 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" event={"ID":"87550db3-1a9e-4b21-a7ae-436618bb9d9c","Type":"ContainerDied","Data":"bca0401738c2b155bfcbf284f8df7b9bc4b518c0fda2293567e395e80b7ec52a"} Jul 21 06:57:11.906174 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:11.906110 2563 scope.go:117] "RemoveContainer" containerID="bca0401738c2b155bfcbf284f8df7b9bc4b518c0fda2293567e395e80b7ec52a" Jul 21 06:57:11.907163 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:11.907135 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:57:11.907237 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:11.907206 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:57:11.907280 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:11.907264 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:57:12.913036 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:12.913002 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" event={"ID":"87550db3-1a9e-4b21-a7ae-436618bb9d9c","Type":"ContainerStarted","Data":"d25f156734ad90cc460d90661b7a8ace1e21b1a6002fe19ff3f52718de785686"} Jul 21 06:57:13.240908 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:13.240809 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:57:13.241236 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:13.241183 2563 patch_prober.go:28] interesting pod/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 21 06:57:13.241494 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:13.241444 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" podUID="87550db3-1a9e-4b21-a7ae-436618bb9d9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 21 06:57:14.240810 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:14.240770 2563 patch_prober.go:28] interesting pod/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 21 06:57:14.241342 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:14.240857 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" podUID="87550db3-1a9e-4b21-a7ae-436618bb9d9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 21 06:57:15.240990 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:15.240955 2563 patch_prober.go:28] interesting pod/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 21 06:57:15.241628 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:15.241570 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" podUID="87550db3-1a9e-4b21-a7ae-436618bb9d9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 21 06:57:16.241544 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:16.241509 2563 patch_prober.go:28] interesting pod/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Jul 21 06:57:16.242179 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:16.242146 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" podUID="87550db3-1a9e-4b21-a7ae-436618bb9d9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Jul 21 06:57:17.244052 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:17.244022 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:57:17.244524 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:17.244462 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:57:17.245090 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:17.245069 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 06:57:17.878409 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:17.878370 2563 patch_prober.go:28] interesting pod/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 06:57:17.878605 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:17.878432 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" podUID="353f759a-e896-4edc-8ec7-9ed803e20655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 06:57:19.957354 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:19.957319 2563 generic.go:356] "Generic (PLEG): container finished" podID="353f759a-e896-4edc-8ec7-9ed803e20655" containerID="4a86e23af42cb42211942af5934289d85b8eafcbd35559da2e8788e0c5c60fd1" exitCode=0 Jul 21 06:57:19.957755 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:19.957394 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" event={"ID":"353f759a-e896-4edc-8ec7-9ed803e20655","Type":"ContainerDied","Data":"4a86e23af42cb42211942af5934289d85b8eafcbd35559da2e8788e0c5c60fd1"} Jul 21 06:57:19.957863 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:19.957847 2563 scope.go:117] "RemoveContainer" containerID="4a86e23af42cb42211942af5934289d85b8eafcbd35559da2e8788e0c5c60fd1" Jul 21 06:57:19.958913 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:19.958884 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:57:19.958984 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:19.958969 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:57:19.959038 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:19.959017 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 21 06:57:20.968370 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:20.968338 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" event={"ID":"353f759a-e896-4edc-8ec7-9ed803e20655","Type":"ContainerStarted","Data":"ffe5319490a63917e9629be3b7d7764ca61f1dcddcb8d024e1cba17db4ba7b6e"} Jul 21 06:57:21.304589 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:21.304504 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:21.304756 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:21.304674 2563 patch_prober.go:28] interesting pod/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Jul 21 06:57:21.304756 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:21.304720 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" podUID="353f759a-e896-4edc-8ec7-9ed803e20655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Jul 21 06:57:22.304709 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:22.304677 2563 patch_prober.go:28] interesting pod/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Jul 21 06:57:22.305124 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:22.304728 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" podUID="353f759a-e896-4edc-8ec7-9ed803e20655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Jul 21 06:57:23.304600 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:23.304567 2563 patch_prober.go:28] interesting pod/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Jul 21 06:57:23.304787 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:23.304631 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" podUID="353f759a-e896-4edc-8ec7-9ed803e20655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Jul 21 06:57:24.303811 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:24.303775 2563 patch_prober.go:28] interesting pod/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Jul 21 06:57:24.303998 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:24.303829 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" podUID="353f759a-e896-4edc-8ec7-9ed803e20655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Jul 21 06:57:25.307748 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:25.307722 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:25.308124 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:25.307966 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 06:57:25.308547 ip-10-0-140-159 kubenswrapper[2563]: I0721 06:57:25.308531 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 07:01:07.670794 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:07.670756 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk"] Jul 21 07:01:07.671360 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:07.671020 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" podUID="87550db3-1a9e-4b21-a7ae-436618bb9d9c" containerName="istio-proxy" containerID="cri-o://d25f156734ad90cc460d90661b7a8ace1e21b1a6002fe19ff3f52718de785686" gracePeriod=30 Jul 21 07:01:12.932408 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:12.932344 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 07:01:13.035792 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.035760 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-credential-socket\") pod \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " Jul 21 07:01:13.035792 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.035799 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-workload-socket\") pod \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " Jul 21 07:01:13.036053 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.035844 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs7x5\" (UniqueName: \"kubernetes.io/projected/87550db3-1a9e-4b21-a7ae-436618bb9d9c-kube-api-access-bs7x5\") pod \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " Jul 21 07:01:13.036053 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.035864 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-envoy\") pod \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " Jul 21 07:01:13.036053 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.035881 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-data\") pod \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " Jul 21 07:01:13.036053 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.035915 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istiod-ca-cert\") pod \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " Jul 21 07:01:13.036053 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.035955 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-podinfo\") pod \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " Jul 21 07:01:13.036053 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.035982 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-workload-certs\") pod \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " Jul 21 07:01:13.036053 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.036038 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-token\") pod \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\" (UID: \"87550db3-1a9e-4b21-a7ae-436618bb9d9c\") " Jul 21 07:01:13.036442 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.036056 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "87550db3-1a9e-4b21-a7ae-436618bb9d9c" (UID: "87550db3-1a9e-4b21-a7ae-436618bb9d9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 07:01:13.036442 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.036175 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "87550db3-1a9e-4b21-a7ae-436618bb9d9c" (UID: "87550db3-1a9e-4b21-a7ae-436618bb9d9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 07:01:13.036442 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.036344 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "87550db3-1a9e-4b21-a7ae-436618bb9d9c" (UID: "87550db3-1a9e-4b21-a7ae-436618bb9d9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 07:01:13.036442 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.036420 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-credential-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:13.036442 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.036421 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "87550db3-1a9e-4b21-a7ae-436618bb9d9c" (UID: "87550db3-1a9e-4b21-a7ae-436618bb9d9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 07:01:13.036442 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.036440 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-workload-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:13.036945 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.036920 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-data" (OuterVolumeSpecName: "istio-data") pod "87550db3-1a9e-4b21-a7ae-436618bb9d9c" (UID: "87550db3-1a9e-4b21-a7ae-436618bb9d9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 07:01:13.038297 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.038263 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87550db3-1a9e-4b21-a7ae-436618bb9d9c-kube-api-access-bs7x5" (OuterVolumeSpecName: "kube-api-access-bs7x5") pod "87550db3-1a9e-4b21-a7ae-436618bb9d9c" (UID: "87550db3-1a9e-4b21-a7ae-436618bb9d9c"). InnerVolumeSpecName "kube-api-access-bs7x5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 07:01:13.038412 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.038346 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "87550db3-1a9e-4b21-a7ae-436618bb9d9c" (UID: "87550db3-1a9e-4b21-a7ae-436618bb9d9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 07:01:13.038930 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.038896 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-token" (OuterVolumeSpecName: "istio-token") pod "87550db3-1a9e-4b21-a7ae-436618bb9d9c" (UID: "87550db3-1a9e-4b21-a7ae-436618bb9d9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 07:01:13.038930 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.038903 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "87550db3-1a9e-4b21-a7ae-436618bb9d9c" (UID: "87550db3-1a9e-4b21-a7ae-436618bb9d9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 07:01:13.137639 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.137605 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs7x5\" (UniqueName: \"kubernetes.io/projected/87550db3-1a9e-4b21-a7ae-436618bb9d9c-kube-api-access-bs7x5\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:13.137639 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.137636 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-envoy\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:13.137639 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.137645 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-data\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:13.137894 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.137653 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istiod-ca-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:13.137894 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.137662 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-podinfo\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:13.137894 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.137669 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87550db3-1a9e-4b21-a7ae-436618bb9d9c-workload-certs\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:13.137894 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.137677 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87550db3-1a9e-4b21-a7ae-436618bb9d9c-istio-token\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:13.307882 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.307786 2563 generic.go:356] "Generic (PLEG): container finished" podID="87550db3-1a9e-4b21-a7ae-436618bb9d9c" containerID="d25f156734ad90cc460d90661b7a8ace1e21b1a6002fe19ff3f52718de785686" exitCode=0 Jul 21 07:01:13.307882 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.307835 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" Jul 21 07:01:13.308108 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.307839 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" event={"ID":"87550db3-1a9e-4b21-a7ae-436618bb9d9c","Type":"ContainerDied","Data":"d25f156734ad90cc460d90661b7a8ace1e21b1a6002fe19ff3f52718de785686"} Jul 21 07:01:13.308108 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.307954 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk" event={"ID":"87550db3-1a9e-4b21-a7ae-436618bb9d9c","Type":"ContainerDied","Data":"0fc6b19efbd32b5c4e32b9573934677127897604f2c6b15139f041d894c7b2bf"} Jul 21 07:01:13.308108 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.307975 2563 scope.go:117] "RemoveContainer" containerID="d25f156734ad90cc460d90661b7a8ace1e21b1a6002fe19ff3f52718de785686" Jul 21 07:01:13.318483 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.318465 2563 scope.go:117] "RemoveContainer" containerID="bca0401738c2b155bfcbf284f8df7b9bc4b518c0fda2293567e395e80b7ec52a" Jul 21 07:01:13.326956 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.326938 2563 scope.go:117] "RemoveContainer" containerID="d25f156734ad90cc460d90661b7a8ace1e21b1a6002fe19ff3f52718de785686" Jul 21 07:01:13.327238 ip-10-0-140-159 kubenswrapper[2563]: E0721 07:01:13.327190 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d25f156734ad90cc460d90661b7a8ace1e21b1a6002fe19ff3f52718de785686\": container with ID starting with d25f156734ad90cc460d90661b7a8ace1e21b1a6002fe19ff3f52718de785686 not found: ID does not exist" containerID="d25f156734ad90cc460d90661b7a8ace1e21b1a6002fe19ff3f52718de785686" Jul 21 07:01:13.327339 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.327319 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d25f156734ad90cc460d90661b7a8ace1e21b1a6002fe19ff3f52718de785686"} err="failed to get container status \"d25f156734ad90cc460d90661b7a8ace1e21b1a6002fe19ff3f52718de785686\": rpc error: code = NotFound desc = could not find container \"d25f156734ad90cc460d90661b7a8ace1e21b1a6002fe19ff3f52718de785686\": container with ID starting with d25f156734ad90cc460d90661b7a8ace1e21b1a6002fe19ff3f52718de785686 not found: ID does not exist" Jul 21 07:01:13.327386 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.327342 2563 scope.go:117] "RemoveContainer" containerID="bca0401738c2b155bfcbf284f8df7b9bc4b518c0fda2293567e395e80b7ec52a" Jul 21 07:01:13.327582 ip-10-0-140-159 kubenswrapper[2563]: E0721 07:01:13.327566 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bca0401738c2b155bfcbf284f8df7b9bc4b518c0fda2293567e395e80b7ec52a\": container with ID starting with bca0401738c2b155bfcbf284f8df7b9bc4b518c0fda2293567e395e80b7ec52a not found: ID does not exist" containerID="bca0401738c2b155bfcbf284f8df7b9bc4b518c0fda2293567e395e80b7ec52a" Jul 21 07:01:13.327624 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.327587 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bca0401738c2b155bfcbf284f8df7b9bc4b518c0fda2293567e395e80b7ec52a"} err="failed to get container status \"bca0401738c2b155bfcbf284f8df7b9bc4b518c0fda2293567e395e80b7ec52a\": rpc error: code = NotFound desc = could not find container \"bca0401738c2b155bfcbf284f8df7b9bc4b518c0fda2293567e395e80b7ec52a\": container with ID starting with bca0401738c2b155bfcbf284f8df7b9bc4b518c0fda2293567e395e80b7ec52a not found: ID does not exist" Jul 21 07:01:13.334738 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.334717 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk"] Jul 21 07:01:13.339871 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:13.339851 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-08eb3e-openshift-default-89b884d99-pk4jk"] Jul 21 07:01:14.486796 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:14.486756 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87550db3-1a9e-4b21-a7ae-436618bb9d9c" path="/var/lib/kubelet/pods/87550db3-1a9e-4b21-a7ae-436618bb9d9c/volumes" Jul 21 07:01:29.680422 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:29.680386 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk"] Jul 21 07:01:29.680831 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:29.680680 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" podUID="353f759a-e896-4edc-8ec7-9ed803e20655" containerName="istio-proxy" containerID="cri-o://ffe5319490a63917e9629be3b7d7764ca61f1dcddcb8d024e1cba17db4ba7b6e" gracePeriod=30 Jul 21 07:01:34.944630 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:34.944566 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 07:01:35.054869 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.054832 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-workload-socket\") pod \"353f759a-e896-4edc-8ec7-9ed803e20655\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " Jul 21 07:01:35.055050 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.054881 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-credential-socket\") pod \"353f759a-e896-4edc-8ec7-9ed803e20655\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " Jul 21 07:01:35.055050 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.054906 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-workload-certs\") pod \"353f759a-e896-4edc-8ec7-9ed803e20655\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " Jul 21 07:01:35.055050 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.054927 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/353f759a-e896-4edc-8ec7-9ed803e20655-istio-podinfo\") pod \"353f759a-e896-4edc-8ec7-9ed803e20655\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " Jul 21 07:01:35.055050 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.054958 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkzgq\" (UniqueName: \"kubernetes.io/projected/353f759a-e896-4edc-8ec7-9ed803e20655-kube-api-access-kkzgq\") pod \"353f759a-e896-4edc-8ec7-9ed803e20655\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " Jul 21 07:01:35.055050 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.054980 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-istio-envoy\") pod \"353f759a-e896-4edc-8ec7-9ed803e20655\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " Jul 21 07:01:35.055050 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.055025 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/353f759a-e896-4edc-8ec7-9ed803e20655-istio-token\") pod \"353f759a-e896-4edc-8ec7-9ed803e20655\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " Jul 21 07:01:35.055418 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.055080 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-istio-data\") pod \"353f759a-e896-4edc-8ec7-9ed803e20655\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " Jul 21 07:01:35.055418 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.055115 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/353f759a-e896-4edc-8ec7-9ed803e20655-istiod-ca-cert\") pod \"353f759a-e896-4edc-8ec7-9ed803e20655\" (UID: \"353f759a-e896-4edc-8ec7-9ed803e20655\") " Jul 21 07:01:35.055418 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.055250 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "353f759a-e896-4edc-8ec7-9ed803e20655" (UID: "353f759a-e896-4edc-8ec7-9ed803e20655"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 07:01:35.055418 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.055265 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "353f759a-e896-4edc-8ec7-9ed803e20655" (UID: "353f759a-e896-4edc-8ec7-9ed803e20655"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 07:01:35.055418 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.055352 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "353f759a-e896-4edc-8ec7-9ed803e20655" (UID: "353f759a-e896-4edc-8ec7-9ed803e20655"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 07:01:35.055667 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.055594 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353f759a-e896-4edc-8ec7-9ed803e20655-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "353f759a-e896-4edc-8ec7-9ed803e20655" (UID: "353f759a-e896-4edc-8ec7-9ed803e20655"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 07:01:35.055667 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.055603 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-workload-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:35.055667 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.055625 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-credential-socket\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:35.055667 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.055641 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-workload-certs\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:35.056419 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.056391 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-istio-data" (OuterVolumeSpecName: "istio-data") pod "353f759a-e896-4edc-8ec7-9ed803e20655" (UID: "353f759a-e896-4edc-8ec7-9ed803e20655"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 07:01:35.057434 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.057402 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/353f759a-e896-4edc-8ec7-9ed803e20655-istio-token" (OuterVolumeSpecName: "istio-token") pod "353f759a-e896-4edc-8ec7-9ed803e20655" (UID: "353f759a-e896-4edc-8ec7-9ed803e20655"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 07:01:35.057434 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.057413 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/353f759a-e896-4edc-8ec7-9ed803e20655-kube-api-access-kkzgq" (OuterVolumeSpecName: "kube-api-access-kkzgq") pod "353f759a-e896-4edc-8ec7-9ed803e20655" (UID: "353f759a-e896-4edc-8ec7-9ed803e20655"). InnerVolumeSpecName "kube-api-access-kkzgq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 07:01:35.057565 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.057412 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/353f759a-e896-4edc-8ec7-9ed803e20655-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "353f759a-e896-4edc-8ec7-9ed803e20655" (UID: "353f759a-e896-4edc-8ec7-9ed803e20655"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 07:01:35.057565 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.057472 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "353f759a-e896-4edc-8ec7-9ed803e20655" (UID: "353f759a-e896-4edc-8ec7-9ed803e20655"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 07:01:35.156587 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.156554 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/353f759a-e896-4edc-8ec7-9ed803e20655-istio-podinfo\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:35.156587 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.156582 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkzgq\" (UniqueName: \"kubernetes.io/projected/353f759a-e896-4edc-8ec7-9ed803e20655-kube-api-access-kkzgq\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:35.156587 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.156594 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-istio-envoy\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:35.156827 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.156602 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/353f759a-e896-4edc-8ec7-9ed803e20655-istio-token\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:35.156827 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.156609 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/353f759a-e896-4edc-8ec7-9ed803e20655-istio-data\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:35.156827 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.156616 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/353f759a-e896-4edc-8ec7-9ed803e20655-istiod-ca-cert\") on node \"ip-10-0-140-159.ec2.internal\" DevicePath \"\"" Jul 21 07:01:35.437575 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.437535 2563 generic.go:356] "Generic (PLEG): container finished" podID="353f759a-e896-4edc-8ec7-9ed803e20655" containerID="ffe5319490a63917e9629be3b7d7764ca61f1dcddcb8d024e1cba17db4ba7b6e" exitCode=0 Jul 21 07:01:35.437761 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.437598 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" Jul 21 07:01:35.437761 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.437618 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" event={"ID":"353f759a-e896-4edc-8ec7-9ed803e20655","Type":"ContainerDied","Data":"ffe5319490a63917e9629be3b7d7764ca61f1dcddcb8d024e1cba17db4ba7b6e"} Jul 21 07:01:35.437761 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.437699 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk" event={"ID":"353f759a-e896-4edc-8ec7-9ed803e20655","Type":"ContainerDied","Data":"bed8b1f20cb22f23b13b82f860b58735883fb0cb82cc6f6bfbcf89339a53ad54"} Jul 21 07:01:35.437761 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.437724 2563 scope.go:117] "RemoveContainer" containerID="ffe5319490a63917e9629be3b7d7764ca61f1dcddcb8d024e1cba17db4ba7b6e" Jul 21 07:01:35.448305 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.448285 2563 scope.go:117] "RemoveContainer" containerID="4a86e23af42cb42211942af5934289d85b8eafcbd35559da2e8788e0c5c60fd1" Jul 21 07:01:35.456649 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.456631 2563 scope.go:117] "RemoveContainer" containerID="ffe5319490a63917e9629be3b7d7764ca61f1dcddcb8d024e1cba17db4ba7b6e" Jul 21 07:01:35.456898 ip-10-0-140-159 kubenswrapper[2563]: E0721 07:01:35.456881 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffe5319490a63917e9629be3b7d7764ca61f1dcddcb8d024e1cba17db4ba7b6e\": container with ID starting with ffe5319490a63917e9629be3b7d7764ca61f1dcddcb8d024e1cba17db4ba7b6e not found: ID does not exist" containerID="ffe5319490a63917e9629be3b7d7764ca61f1dcddcb8d024e1cba17db4ba7b6e" Jul 21 07:01:35.456939 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.456907 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe5319490a63917e9629be3b7d7764ca61f1dcddcb8d024e1cba17db4ba7b6e"} err="failed to get container status \"ffe5319490a63917e9629be3b7d7764ca61f1dcddcb8d024e1cba17db4ba7b6e\": rpc error: code = NotFound desc = could not find container \"ffe5319490a63917e9629be3b7d7764ca61f1dcddcb8d024e1cba17db4ba7b6e\": container with ID starting with ffe5319490a63917e9629be3b7d7764ca61f1dcddcb8d024e1cba17db4ba7b6e not found: ID does not exist" Jul 21 07:01:35.456939 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.456921 2563 scope.go:117] "RemoveContainer" containerID="4a86e23af42cb42211942af5934289d85b8eafcbd35559da2e8788e0c5c60fd1" Jul 21 07:01:35.457124 ip-10-0-140-159 kubenswrapper[2563]: E0721 07:01:35.457108 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a86e23af42cb42211942af5934289d85b8eafcbd35559da2e8788e0c5c60fd1\": container with ID starting with 4a86e23af42cb42211942af5934289d85b8eafcbd35559da2e8788e0c5c60fd1 not found: ID does not exist" containerID="4a86e23af42cb42211942af5934289d85b8eafcbd35559da2e8788e0c5c60fd1" Jul 21 07:01:35.457170 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.457129 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a86e23af42cb42211942af5934289d85b8eafcbd35559da2e8788e0c5c60fd1"} err="failed to get container status \"4a86e23af42cb42211942af5934289d85b8eafcbd35559da2e8788e0c5c60fd1\": rpc error: code = NotFound desc = could not find container \"4a86e23af42cb42211942af5934289d85b8eafcbd35559da2e8788e0c5c60fd1\": container with ID starting with 4a86e23af42cb42211942af5934289d85b8eafcbd35559da2e8788e0c5c60fd1 not found: ID does not exist" Jul 21 07:01:35.463917 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.463887 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk"] Jul 21 07:01:35.467126 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.467105 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-bddf64-openshift-default-7849569467-xtqsk"] Jul 21 07:01:35.774361 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.774288 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-99gxr_d95d08d0-1a92-457d-98d8-6e347fe04d32/manager/0.log" Jul 21 07:01:35.889595 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:35.889561 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-w6btv_ce0b4c40-cbda-4526-8404-69090e199629/manager/0.log" Jul 21 07:01:36.264736 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:36.264709 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gcnl9_c821d0c2-52a7-4253-9dfe-1b6e7dc709ae/manager/0.log" Jul 21 07:01:36.380929 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:36.380902 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c844c949-9v75q_b494fd0f-dff8-4843-8e75-2b326e9b2323/copy-manifests/0.log" Jul 21 07:01:36.387717 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:36.387690 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c844c949-9v75q_b494fd0f-dff8-4843-8e75-2b326e9b2323/manager/0.log" Jul 21 07:01:36.487064 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:36.487035 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="353f759a-e896-4edc-8ec7-9ed803e20655" path="/var/lib/kubelet/pods/353f759a-e896-4edc-8ec7-9ed803e20655/volumes" Jul 21 07:01:37.097445 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:37.097420 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743590-59s7g_d41f0c3d-6fd5-454b-9f40-9c4abeb088fb/cleanup/0.log" Jul 21 07:01:38.208035 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:38.208006 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5_1cad27d4-5fce-4a5e-b2a5-100d1ae5632d/pull/0.log" Jul 21 07:01:38.214583 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:38.214563 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5_1cad27d4-5fce-4a5e-b2a5-100d1ae5632d/extract/0.log" Jul 21 07:01:38.220653 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:38.220633 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5_1cad27d4-5fce-4a5e-b2a5-100d1ae5632d/util/0.log" Jul 21 07:01:38.336073 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:38.336046 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn_a1e61a69-b7fe-4663-ba0f-451a52b630af/util/0.log" Jul 21 07:01:38.342738 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:38.342720 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn_a1e61a69-b7fe-4663-ba0f-451a52b630af/pull/0.log" Jul 21 07:01:38.349477 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:38.349460 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn_a1e61a69-b7fe-4663-ba0f-451a52b630af/extract/0.log" Jul 21 07:01:38.470201 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:38.470134 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx_f305a214-c729-4ed3-b051-3611cc3b061d/pull/0.log" Jul 21 07:01:38.475658 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:38.475641 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx_f305a214-c729-4ed3-b051-3611cc3b061d/extract/0.log" Jul 21 07:01:38.481088 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:38.481061 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx_f305a214-c729-4ed3-b051-3611cc3b061d/util/0.log" Jul 21 07:01:38.592428 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:38.592399 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4_011af3cc-80f2-4240-9519-fc4dc072d7e9/util/0.log" Jul 21 07:01:38.598296 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:38.598276 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4_011af3cc-80f2-4240-9519-fc4dc072d7e9/pull/0.log" Jul 21 07:01:38.603634 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:38.603611 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4_011af3cc-80f2-4240-9519-fc4dc072d7e9/extract/0.log" Jul 21 07:01:39.314854 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:39.314826 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-kvnrx_66391c1a-0c11-4fef-b90a-715f00d51683/manager/0.log" Jul 21 07:01:39.914499 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:39.914472 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rxdmc_cbe60fb0-7883-43a9-be69-39cfaa4b3c5c/istio-proxy/0.log" Jul 21 07:01:40.031160 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:40.031131 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2dvfs_4f535186-b6ab-49c5-90e3-a629cfbed20f/discovery/0.log" Jul 21 07:01:40.268138 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:40.268057 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ff9c87565-srhn6_5418a32d-6c58-4113-8ce4-52753a3d4401/kube-auth-proxy/0.log" Jul 21 07:01:40.390302 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:40.390274 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-9k8mj_1c492a72-9f73-46e1-b8df-4e771849cccf/istio-proxy/1.log" Jul 21 07:01:40.510410 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:40.510379 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mpdhh_4dee74c0-b8f6-40ac-ae92-048eef2fa77f/payload-pre-processing/0.log" Jul 21 07:01:41.221990 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:41.221960 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-826h5_6be1d497-31f8-4579-b75c-ff2a57a6d1b0/main/0.log" Jul 21 07:01:41.329877 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:41.329846 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-lh8nw_dafddc54-e5bd-4a22-a334-c4b11c61dd27/main/0.log" Jul 21 07:01:49.871344 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:49.871314 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kwzdp_2ee4a567-fe9e-4346-ac51-1d78dede57d1/global-pull-secret-syncer/0.log" Jul 21 07:01:50.074927 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:50.074893 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nzgkk_dd4adafa-b828-4ddc-8668-036cb8960d77/konnectivity-agent/0.log" Jul 21 07:01:50.137058 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:50.136979 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-159.ec2.internal_0dad6f1e84139608ab928a8fbb218961/haproxy/0.log" Jul 21 07:01:54.808632 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:54.808599 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5_1cad27d4-5fce-4a5e-b2a5-100d1ae5632d/extract/0.log" Jul 21 07:01:54.836583 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:54.836556 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5_1cad27d4-5fce-4a5e-b2a5-100d1ae5632d/util/0.log" Jul 21 07:01:54.856969 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:54.856946 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759v84j5_1cad27d4-5fce-4a5e-b2a5-100d1ae5632d/pull/0.log" Jul 21 07:01:54.892023 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:54.891990 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn_a1e61a69-b7fe-4663-ba0f-451a52b630af/extract/0.log" Jul 21 07:01:54.912513 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:54.912490 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn_a1e61a69-b7fe-4663-ba0f-451a52b630af/util/0.log" Jul 21 07:01:54.937478 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:54.937460 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04dqvn_a1e61a69-b7fe-4663-ba0f-451a52b630af/pull/0.log" Jul 21 07:01:54.963942 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:54.963922 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx_f305a214-c729-4ed3-b051-3611cc3b061d/extract/0.log" Jul 21 07:01:54.992538 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:54.992515 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx_f305a214-c729-4ed3-b051-3611cc3b061d/util/0.log" Jul 21 07:01:55.011832 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:55.011815 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73zpbhx_f305a214-c729-4ed3-b051-3611cc3b061d/pull/0.log" Jul 21 07:01:55.039060 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:55.039040 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4_011af3cc-80f2-4240-9519-fc4dc072d7e9/extract/0.log" Jul 21 07:01:55.062456 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:55.062396 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4_011af3cc-80f2-4240-9519-fc4dc072d7e9/util/0.log" Jul 21 07:01:55.083498 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:55.083473 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mxkc4_011af3cc-80f2-4240-9519-fc4dc072d7e9/pull/0.log" Jul 21 07:01:55.748881 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:55.748851 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-kvnrx_66391c1a-0c11-4fef-b90a-715f00d51683/manager/0.log" Jul 21 07:01:57.564282 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:57.564200 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_17c2cf80-79a4-4fc4-8a15-0cc6354d405a/alertmanager/0.log" Jul 21 07:01:57.591454 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:57.591436 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_17c2cf80-79a4-4fc4-8a15-0cc6354d405a/config-reloader/0.log" Jul 21 07:01:57.638457 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:57.638435 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_17c2cf80-79a4-4fc4-8a15-0cc6354d405a/kube-rbac-proxy-web/0.log" Jul 21 07:01:57.671700 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:57.671665 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_17c2cf80-79a4-4fc4-8a15-0cc6354d405a/kube-rbac-proxy/0.log" Jul 21 07:01:57.710172 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:57.710144 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_17c2cf80-79a4-4fc4-8a15-0cc6354d405a/kube-rbac-proxy-metric/0.log" Jul 21 07:01:57.736505 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:57.736484 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_17c2cf80-79a4-4fc4-8a15-0cc6354d405a/prom-label-proxy/0.log" Jul 21 07:01:57.766615 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:57.766592 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_17c2cf80-79a4-4fc4-8a15-0cc6354d405a/init-config-reloader/0.log" Jul 21 07:01:57.818086 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:57.818024 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-dggkd_e090c63e-385e-432f-a56f-b3601847ad54/cluster-monitoring-operator/0.log" Jul 21 07:01:57.843572 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:57.843548 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-7f792_c3bc626c-3b7d-4689-baa0-4c22c56360d3/kube-state-metrics/0.log" Jul 21 07:01:57.870873 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:57.870854 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-7f792_c3bc626c-3b7d-4689-baa0-4c22c56360d3/kube-rbac-proxy-main/0.log" Jul 21 07:01:57.900560 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:57.900541 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-7f792_c3bc626c-3b7d-4689-baa0-4c22c56360d3/kube-rbac-proxy-self/0.log" Jul 21 07:01:58.019971 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:58.019945 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gqqrw_a7ea94d1-3d5b-4f1e-a587-b09b12fedeae/node-exporter/0.log" Jul 21 07:01:58.051156 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:58.051129 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gqqrw_a7ea94d1-3d5b-4f1e-a587-b09b12fedeae/kube-rbac-proxy/0.log" Jul 21 07:01:58.075446 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:58.075424 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gqqrw_a7ea94d1-3d5b-4f1e-a587-b09b12fedeae/init-textfile/0.log" Jul 21 07:01:58.294451 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:58.294421 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-5v4f7_4eb512c1-6716-43c5-a14b-c88e0a2eaf14/kube-rbac-proxy-main/0.log" Jul 21 07:01:58.320810 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:58.320787 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-5v4f7_4eb512c1-6716-43c5-a14b-c88e0a2eaf14/kube-rbac-proxy-self/0.log" Jul 21 07:01:58.343861 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:58.343788 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-5v4f7_4eb512c1-6716-43c5-a14b-c88e0a2eaf14/openshift-state-metrics/0.log" Jul 21 07:01:58.587925 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:58.587899 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-zrffr_c55958ac-8a82-471d-8af9-90012d8aee16/prometheus-operator-admission-webhook/0.log" Jul 21 07:01:58.702818 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:58.702744 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668cc967ff-j4bdq_71c1f691-c0f4-486e-abdd-1d88f97bef96/thanos-query/0.log" Jul 21 07:01:58.723742 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:58.723720 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668cc967ff-j4bdq_71c1f691-c0f4-486e-abdd-1d88f97bef96/kube-rbac-proxy-web/0.log" Jul 21 07:01:58.744008 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:58.743973 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668cc967ff-j4bdq_71c1f691-c0f4-486e-abdd-1d88f97bef96/kube-rbac-proxy/0.log" Jul 21 07:01:58.764207 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:58.764185 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668cc967ff-j4bdq_71c1f691-c0f4-486e-abdd-1d88f97bef96/prom-label-proxy/0.log" Jul 21 07:01:58.784330 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:58.784308 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668cc967ff-j4bdq_71c1f691-c0f4-486e-abdd-1d88f97bef96/kube-rbac-proxy-rules/0.log" Jul 21 07:01:58.805103 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:01:58.805084 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668cc967ff-j4bdq_71c1f691-c0f4-486e-abdd-1d88f97bef96/kube-rbac-proxy-metrics/0.log" Jul 21 07:02:00.437336 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.437304 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc"] Jul 21 07:02:00.437820 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.437801 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87550db3-1a9e-4b21-a7ae-436618bb9d9c" containerName="istio-proxy" Jul 21 07:02:00.437868 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.437823 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="87550db3-1a9e-4b21-a7ae-436618bb9d9c" containerName="istio-proxy" Jul 21 07:02:00.437868 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.437835 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="353f759a-e896-4edc-8ec7-9ed803e20655" containerName="istio-proxy" Jul 21 07:02:00.437868 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.437840 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="353f759a-e896-4edc-8ec7-9ed803e20655" containerName="istio-proxy" Jul 21 07:02:00.437959 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.437916 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="353f759a-e896-4edc-8ec7-9ed803e20655" containerName="istio-proxy" Jul 21 07:02:00.437959 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.437926 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="87550db3-1a9e-4b21-a7ae-436618bb9d9c" containerName="istio-proxy" Jul 21 07:02:00.437959 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.437932 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="353f759a-e896-4edc-8ec7-9ed803e20655" containerName="istio-proxy" Jul 21 07:02:00.437959 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.437938 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="87550db3-1a9e-4b21-a7ae-436618bb9d9c" containerName="istio-proxy" Jul 21 07:02:00.441176 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.441152 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.444189 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.444167 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mtl9f\"/\"openshift-service-ca.crt\"" Jul 21 07:02:00.444347 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.444312 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mtl9f\"/\"kube-root-ca.crt\"" Jul 21 07:02:00.444347 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.444322 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mtl9f\"/\"default-dockercfg-qq7rw\"" Jul 21 07:02:00.450068 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.450017 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc"] Jul 21 07:02:00.495599 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.495574 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-proc\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.495753 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.495608 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-podres\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.495753 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.495642 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc98g\" (UniqueName: \"kubernetes.io/projected/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-kube-api-access-kc98g\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.495753 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.495704 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-sys\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.495865 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.495769 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-lib-modules\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.596389 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.596360 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc98g\" (UniqueName: \"kubernetes.io/projected/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-kube-api-access-kc98g\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.596389 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.596388 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-sys\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.596580 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.596423 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-lib-modules\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.596580 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.596487 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-sys\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.596580 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.596538 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-proc\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.596580 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.596568 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-lib-modules\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.596766 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.596633 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-proc\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.596766 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.596716 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-podres\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.596857 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.596842 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-podres\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.604752 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.604727 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc98g\" (UniqueName: \"kubernetes.io/projected/b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1-kube-api-access-kc98g\") pod \"perf-node-gather-daemonset-ptfrc\" (UID: \"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1\") " pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.754239 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.754137 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:00.890808 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:00.890779 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc"] Jul 21 07:02:00.892604 ip-10-0-140-159 kubenswrapper[2563]: W0721 07:02:00.892572 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb9f39f0d_3c24_4404_bd7a_d6b5d6b46fe1.slice/crio-f37c01b1e1971c54f4755ce4820638851fbf714db7da282a8ea6c0fa95186dee WatchSource:0}: Error finding container f37c01b1e1971c54f4755ce4820638851fbf714db7da282a8ea6c0fa95186dee: Status 404 returned error can't find the container with id f37c01b1e1971c54f4755ce4820638851fbf714db7da282a8ea6c0fa95186dee Jul 21 07:02:01.588892 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:01.588857 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" event={"ID":"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1","Type":"ContainerStarted","Data":"00f87ff71e14296c7298de008ea564917e5ccf150ed27f931c1732d4a564cc03"} Jul 21 07:02:01.589416 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:01.588911 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" event={"ID":"b9f39f0d-3c24-4404-bd7a-d6b5d6b46fe1","Type":"ContainerStarted","Data":"f37c01b1e1971c54f4755ce4820638851fbf714db7da282a8ea6c0fa95186dee"} Jul 21 07:02:01.589416 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:01.588993 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:01.604808 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:01.604762 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" podStartSLOduration=1.6047482130000001 podStartE2EDuration="1.604748213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 07:02:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 07:02:01.604503147 +0000 UTC m=+2651.550141804" watchObservedRunningTime="2026-07-21 07:02:01.604748213 +0000 UTC m=+2651.550386869" Jul 21 07:02:02.808583 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:02.808554 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2gh49_8b9dd21a-f6c5-478c-8dc5-024a0fb145f2/dns/0.log" Jul 21 07:02:02.826875 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:02.826852 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2gh49_8b9dd21a-f6c5-478c-8dc5-024a0fb145f2/kube-rbac-proxy/0.log" Jul 21 07:02:02.961331 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:02.961297 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ks8rh_512070fa-797a-4c81-92f3-0510a4c7cfbb/dns-node-resolver/0.log" Jul 21 07:02:03.511380 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:03.511347 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-554659c859-lmjbc_1db48e88-ac22-4412-bfd7-cd4fde7a37e8/registry/0.log" Jul 21 07:02:03.530830 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:03.530805 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6dtks_7b9eedde-6415-425e-9223-d74b2a3f74cb/node-ca/0.log" Jul 21 07:02:04.508789 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:04.508764 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rxdmc_cbe60fb0-7883-43a9-be69-39cfaa4b3c5c/istio-proxy/0.log" Jul 21 07:02:04.628024 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:04.627988 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2dvfs_4f535186-b6ab-49c5-90e3-a629cfbed20f/discovery/0.log" Jul 21 07:02:04.671008 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:04.670983 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ff9c87565-srhn6_5418a32d-6c58-4113-8ce4-52753a3d4401/kube-auth-proxy/0.log" Jul 21 07:02:04.698539 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:04.698513 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-9k8mj_1c492a72-9f73-46e1-b8df-4e771849cccf/istio-proxy/0.log" Jul 21 07:02:04.722391 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:04.722368 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-9k8mj_1c492a72-9f73-46e1-b8df-4e771849cccf/istio-proxy/1.log" Jul 21 07:02:04.750839 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:04.750811 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mpdhh_4dee74c0-b8f6-40ac-ae92-048eef2fa77f/payload-pre-processing/0.log" Jul 21 07:02:05.557625 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:05.557596 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w6zct_c253498b-ea18-4b4d-b21d-116381a32d7f/serve-healthcheck-canary/0.log" Jul 21 07:02:06.031627 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:06.031596 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-pccmp_dc976a1b-fc68-43d0-ad64-2b6407be665c/insights-operator/0.log" Jul 21 07:02:06.174058 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:06.173985 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zzv9d_05fea74d-5017-4bf7-b70b-70919bb2c4a6/kube-rbac-proxy/0.log" Jul 21 07:02:06.192769 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:06.192742 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zzv9d_05fea74d-5017-4bf7-b70b-70919bb2c4a6/exporter/0.log" Jul 21 07:02:06.212636 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:06.212620 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zzv9d_05fea74d-5017-4bf7-b70b-70919bb2c4a6/extractor/0.log" Jul 21 07:02:07.607840 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:07.607817 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mtl9f/perf-node-gather-daemonset-ptfrc" Jul 21 07:02:08.427393 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:08.427362 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-99gxr_d95d08d0-1a92-457d-98d8-6e347fe04d32/manager/0.log" Jul 21 07:02:08.587651 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:08.587621 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-w6btv_ce0b4c40-cbda-4526-8404-69090e199629/manager/0.log" Jul 21 07:02:08.859199 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:08.859162 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-gcnl9_c821d0c2-52a7-4253-9dfe-1b6e7dc709ae/manager/0.log" Jul 21 07:02:08.884935 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:08.884909 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c844c949-9v75q_b494fd0f-dff8-4843-8e75-2b326e9b2323/manager/0.log" Jul 21 07:02:08.905384 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:08.905363 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c844c949-9v75q_b494fd0f-dff8-4843-8e75-2b326e9b2323/copy-manifests/0.log" Jul 21 07:02:10.961366 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:10.961281 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rxdmc_cbe60fb0-7883-43a9-be69-39cfaa4b3c5c/istio-proxy/0.log" Jul 21 07:02:11.069096 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:11.069066 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2dvfs_4f535186-b6ab-49c5-90e3-a629cfbed20f/discovery/0.log" Jul 21 07:02:11.097186 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:11.097152 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ff9c87565-srhn6_5418a32d-6c58-4113-8ce4-52753a3d4401/kube-auth-proxy/0.log" Jul 21 07:02:11.115049 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:11.115025 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-9k8mj_1c492a72-9f73-46e1-b8df-4e771849cccf/istio-proxy/0.log" Jul 21 07:02:11.138056 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:11.138028 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-9k8mj_1c492a72-9f73-46e1-b8df-4e771849cccf/istio-proxy/1.log" Jul 21 07:02:11.155121 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:11.155097 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mpdhh_4dee74c0-b8f6-40ac-ae92-048eef2fa77f/payload-pre-processing/0.log" Jul 21 07:02:12.429203 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:12.429172 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rxdmc_cbe60fb0-7883-43a9-be69-39cfaa4b3c5c/istio-proxy/0.log" Jul 21 07:02:12.541422 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:12.541391 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2dvfs_4f535186-b6ab-49c5-90e3-a629cfbed20f/discovery/0.log" Jul 21 07:02:12.572293 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:12.572269 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ff9c87565-srhn6_5418a32d-6c58-4113-8ce4-52753a3d4401/kube-auth-proxy/0.log" Jul 21 07:02:12.600641 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:12.600615 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-9k8mj_1c492a72-9f73-46e1-b8df-4e771849cccf/istio-proxy/0.log" Jul 21 07:02:12.623880 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:12.623853 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-9k8mj_1c492a72-9f73-46e1-b8df-4e771849cccf/istio-proxy/1.log" Jul 21 07:02:12.643982 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:12.643962 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-mpdhh_4dee74c0-b8f6-40ac-ae92-048eef2fa77f/payload-pre-processing/0.log" Jul 21 07:02:15.539904 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:15.539876 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-9prh8_e7478004-cb1c-4941-a111-d7e5bc4e308d/migrator/0.log" Jul 21 07:02:15.564208 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:15.564183 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-9prh8_e7478004-cb1c-4941-a111-d7e5bc4e308d/graceful-termination/0.log" Jul 21 07:02:16.912257 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:16.912204 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gzpql_cf0536e7-62ba-4202-99af-2d8c13d615f2/kube-multus-additional-cni-plugins/0.log" Jul 21 07:02:16.933020 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:16.932993 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gzpql_cf0536e7-62ba-4202-99af-2d8c13d615f2/egress-router-binary-copy/0.log" Jul 21 07:02:16.951452 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:16.951435 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gzpql_cf0536e7-62ba-4202-99af-2d8c13d615f2/cni-plugins/0.log" Jul 21 07:02:16.971406 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:16.971392 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gzpql_cf0536e7-62ba-4202-99af-2d8c13d615f2/bond-cni-plugin/0.log" Jul 21 07:02:16.991928 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:16.991911 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gzpql_cf0536e7-62ba-4202-99af-2d8c13d615f2/routeoverride-cni/0.log" Jul 21 07:02:17.012607 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:17.012585 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gzpql_cf0536e7-62ba-4202-99af-2d8c13d615f2/whereabouts-cni-bincopy/0.log" Jul 21 07:02:17.038926 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:17.038902 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gzpql_cf0536e7-62ba-4202-99af-2d8c13d615f2/whereabouts-cni/0.log" Jul 21 07:02:17.415746 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:17.415718 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fkwwc_88a16c6f-e274-49dd-952e-065d7ba5b89c/kube-multus/0.log" Jul 21 07:02:17.485565 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:17.485539 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l2867_42908595-6e16-4688-b583-504934328eb7/network-metrics-daemon/0.log" Jul 21 07:02:17.507550 ip-10-0-140-159 kubenswrapper[2563]: I0721 07:02:17.507529 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l2867_42908595-6e16-4688-b583-504934328eb7/kube-rbac-proxy/0.log"